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ABSTRACT 


This  thesis  assesses  the  capability  of  the  Logistics 
Management  Decision  Support  System  (LMDSS)  to  meet  the 
information  needs  of  Naval  Air  Systems  Command  (NAVAIR) 
logistics  managers  based  on  surveys  of  logistics  managers 
and  interviews  with  LMDSS  program  representatives. 

The  LMDSS  is  being  introduced  as  a  tool  to  facilitate 
action  by  NAVAIR  logistics  managers  to  reduce  the  life  cycle 
support  costs  of  aviation  systems  while  protecting 
readiness.  We  conclude  the  LMDSS  does  not  meet  the 
definition  of  a  Decision  Support  System  due  to  the  lack  of 
modeling  capabilities.  The  LMDSS  architecture  and 
capabilities  meet  the  information  needs  of  surveyed 
logistics  managers  and  support  Affordable  Readiness 
initiatives  which  are  the  means  by  which  NAVAIR  intends  to 
reduce  life  cycle  costs  while  sustaining  aviation  system 
readiness  levels.  Lack  of  modeling,  graphics  and 
sensitivity  analysis  capabilities  limits  identification, 
analysis  and  comparison  of  Affordable  Readiness  initiatives. 

We  recommend  modeling  tools  and  graphics  capabilities 
be  incorporated  as  part  of  the  LMDSS  application.  We 
further  recommend  that  initiatives  to  improve  data  validity 
be  expedited  and  that  Maintenance  Level  3  detail  cost  data 
be  provided.  Recommendations  are  made  for  further  research. 
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I. 


INTRODUCTION 


The  Logistics  Management  Decision  Support  System 
(LMDSS)  is  being  introduced  to  help  Naval  Air  Systems 
Command  (NAVAIR)  logistics  management  teams  reduce  the  life 
cycle  support  costs  of  aviation  systems  while  protecting 
readiness.  Recently  LMDSS  identified  a  "bad  actor"  in  the 
F-14  aircraft  avionics  system.  It  isolated  the  Central 
Signal  Data  Converter  (CSDC)  and  identified  a  replacement 
with  similar  functionality  at  a  lower  cost  over  the  life  of 
the  avionics  computer.  A  former  F-14  APML  stated  without 
hesitation  that  the  $42  million  cost  avoidance  could  not 
have  been  found  without  the  LMDSS  prototype  system  in  place 
(NAVAIR  7.0,  1997)  Was  this  an  isolated  incident?  Can  we 
expect  outcomes  like  this  in  the  future  from  the  LMDSS? 

Operating  and  Support  (O&S)  costs,  that  account  for  50 
to  60  %  of  the  Navy' s  Total  Obligation  Authority,  are 
escalating  as  aircraft  age  and  compete  for  the  same  limited 
resources.  (NAVAIR  TEAM,  1998)  The  Navy  must  improve 
business  processes  to  reduce  costs  in  order  to  secure  the 
resources  needed  for  investments  in  modernization  and 
recapitalization.  Cost-reduction  initiatives  are  driving 
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program  managers  to  treat  O&S  costs  equal  to  other 
performance  criteria. 

Reducing  life  cycle  support  costs  depends  upon 
effective  logistics  management  and  planning  that,  in  turn, 
depends  upon  tools  to  support  decision-making.  The  LMDSS  is 
a  tool  to  reduce  life  cycle  support  costs  of  aviation 
systems.  It  is  a  Naval  Aviation  Logistics  Data  and  Analysis 
(NALDA)  Phase  II  application  that  incorporates  data  from 
existing  maintenance,  flight,  cost  and  material  databases 
into  a  structured  decision  making  process. 
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II.  NAVAL  AVIATION  PROGRAM  MANAGEMENT 
AND  LOGISTICS  DECISION  PROCESSES 


A.  RESPONDING  TO  THE  POST  COLD  WAR  PERIOD 

The  LMDSS  is  a  tool  designed  to  support  the  Program 
Managers,  Air  (PMAs)  reduce  life  cycle  support  costs  by 
supporting  decision-making.  The  Navy  must  find  ways  to 
improve  business  processes  to  reduce  costs  in  order  to 
secure  resources  necessary  to  make  investments  in 
modernization  and  recapitalization  required  to  carry  out 
future  missions.^  NAVAIR  is  responding  to  the  post  cold  war 
period  in  several  ways  (NAVAIR  3. 6. 1.1,  1998).  First,  by 
eliminating  military-unique  requirements  and  procedures  that 
drive  up  acquisition  costs  the  cost  of  aircraft  systems  and 
associated  support  is  reduced.  Second,  new  policies  are 
removing  the  impediments  to  getting  state-of-the-art 
technology  into  Navy  aircraft  and  related  systems.  Advanced 
technology  has  proven  a  true  force  multiplier  in  the 
development  and  deployment  of  weapons  systems  (Hickock, 

1997;  Fox,  1997).  .  Third,  firms  that  traditionally  produced 

^  See  www.acq-ref.navy.mil/pdf/abc.pdf.  Navy  Acquisition 
Reform  for  additional  readings  on  modernization  and 
recapitalization  efforts. 
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goods  primarily,  if  not  solely,  for  the  Department  of 
Defense  are  encouraged  to  diversify  into  commercial  markets. 
Fourth,  new  policies  and  strategies  are  targeting  the 
reduction  of  operating  and  support  (O&S)  costs.  The  large 
consumption  of  resources  in  this  area  threatens  Navy 
modernization  and  recapitalization  efforts  (Hickock,  1997; 
Fox,  1997). 

O&S  costs  account  for  between  50  and  60  %  of  the  Navy's 
Total  Obligation  Authority  (NAVAIR  TEAM,  1998).  These  costs 
are  escalating  as  aircraft  age,  competing  with  investment 
requirements  for  the  same  limited  resources  and  thereby 
hindering  improvements  to  infrastructure.  Cost-reduction 
initiatives  are  changing  the  focus  of  program  managers  who 
must  now  treat  O&S  costs  as  they  do  any  other  performance 
criteria;  systems  must  be  affordable  and  supportable  as  well 
as  meet  operational  requirements. 

B .  AFFORDABLE  READINESS 

Affordable  Readiness  is  the  means  by  which  NAVAIR 
intends  to  significantly  reduce  O&S  costs  while  sustaining 
requisite  readiness  levels.  The  resulting  cost  savings  and 
cost  avoidances  can  then  be  directed  toward  modernization 
and  recapitalization.  The  objective  is  to  meet  required 
mission  performance  and  ensure  safe  operations  at  the  lowest 
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ownership  cost.  Previously  the  focus  of  program  managers 
was  centered  on  schedule  and  the  projected  average  unit 
procurement  cost  with  secondary  interest  on  projected 
operations  and  support  objectives.  The  shift  from  Design  to 
Cost  to  Cost  As  an  Independent  Variable  is  a  philosophical 
shift.  The  previous  approach  resulted  in  maximized 
performance  at  nearly  any  cost.^  In  general,  ownership 
costs  can  be  measured  in  terms  of  manpower,  materials  and 
resources.  Readiness,  availability,  operating  time,  turn¬ 
around-time,  and  other  similar  metrics  measure  performance. 
Proposed  Affordable  Readiness  Metrics  are  listed  in  Figure 
1. 

Affordable  Readiness  is  a  business  practice  with  four 
inter-related  elements:  flexible  sustainment,  sustained 
maintenance  planning,  right sourcing,  and  total  cost  of 
ownership.  Appendix  A  describes  each  element.  Analysis  of 
naval  aviation  O&S  costs  reveals  six  major  drivers  that  the 
program  management  team  can  influence  by  implementing 
Affordable  Readiness:  maintenance  concept,  inventory, 
manpower,  technical  data,  infrastructure,  and  warranties 
(NAVAIR  3. 6. 1.1,  1998) .  Continuous  review  of  in-service 
weapons  systems  to  adjust  the  maintenance  structure  based  on 

2  See  Land,  1997;  Kausal,  1996  for  a  historic  perspective  on 
CAIV. 
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Cost  Reduction  Areas 
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Figure  1  Affordable  Readiness  Proposed  Metrics.  After  NAVAIR  3.6,  Affordable  Readiness 
Brief,  June  1997  and  Affordable  Readiness  Web  Site 


fleet  feedback  concurrently  optimizes  operational 
requirements  and  provides  opportunities  to  eliminate 
unnecessary  costly  activities.  Reliability  Centered 
Maintenance  (RCM)^  analysis  and  Logistics  Engineering  Change 
Proposals  (LECPs)  processes  help  to  achieve  better  inherent 
reliability,  target  technology  insertions,  and  avoid 
obsolescence.  Better  inventory  management  and  repair 
process  analysis  can  reduce  out  of  service  time  for 
aircraft,  spares,  and  support  equipment.  Smart  decisions 
early  in  the  acquisition  planning  process  can  reduce 
material  and  manpower  requirements  to  support  an  aircraft 
system  throughout  its  total  life  cycle.  Partnerships  with 
industry,  consolidating  capabilities,  the  use  of  digitized 
data,  single  process  initiatives,  reinvention  initiatives, 
reliability  warranties,  and  integrated  diagnostics  are  some 
of  the  many  cost-effective  initiatives.  Program  managers 
must  make  intelligent  trade-offs  between  performance  and 
life  cycle  costs.  The  decision  support  systems  must  support 
the  PMA  developing  and  analyzing  alternatives -• 


3  See  www.nalda.navy.mil,  NAVAIR  Logistics,  Affordable 
Readiness  Link. 
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C.  PROGRAM  MANAGER  ROLE  IN  AFFORDABLE  READINESS 

Figure  2  depicts  organization  responsibilities  as  they 
pertain  to  Affordable  Readiness.  Affordable  Readiness  is  a 
management  approach  being  implemented  within  the  existing 
organization  structure.^  The  PMAs  are  responsible  for 
developing  plans  to  implement  and  execute  Affordable 
Readiness;  identifying  specific  reduction  targets  and 
metrics  for  tracking  progress;  setting  priorities  and  making 
investment  trade-offs;  directing  the  actions  of  supporting 
teams  like  Fleet  Support  Teams  and  Integrated  Product  Teams; 
interfacing  with  support  environments  such  as  policy, 
process,  facility  and  infrastructure  organizations  to 
develop  optimal  policies  and  processes;  reporting  actions 
taken  and  results;  and  obtaining  fleet  feedback  on  how  the 
system  is  performing.  Assistant  Program  Managers,  Logistics 
(APMLs)  advise  PMAs  on  logistics  matters.  The  program 
management  office  is  where  the  rubber  meets  the  road  in  life 
cycle  management  and  total  cost  of  ownership  analyses.  It 
is  here  that  the  manager  who  has  both  authority  and 
responsibility  is  held  accountable  for  effective  and 
efficient  allocation  of  resources.  The  PMA  must  balance 

^  See  www.navair.navy.mil,  NAVAIR  for  additional  readings  on 
the  NAVAIR  TEAM  history  and  organization. 
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short  and  long-term  objectives  and  vie  for  critically 
limited  resources.  Additionally,  he  must  do  so  while 
working  within  a  labyrinth  tangled  with  political  and 
bureaucratic  processes  and  pressures. 

D.  INTEGRATED  PRODUCT  AND  PROCESS  DEVELOPMENT 

PMAs  manage  within  the  context  of  Integrated  Product 
and  Process  Development  (IPPD)  which  is  a  management  process 
that  integrates  all  activities  from  product  concept  through 
product ion/field  support.  It  uses  a  multi-functional  team 
to  optimize  the  product  and  its  manufacturing  and 
sustainment  simultaneously  to  meet  cost  and  performance 
objectives.  IPPD  evolved  from  concurrent  engineering  and 
the  philosophies  of  quality  management.  It  is  a  system 
engineering  process  integrated  with  sound  business  practices 
and  common  sense  decision  making. 

Integrated  Product  Teams  (IPTs)  are  the  means  through 
which  IPPD  is  implemented.  Appendix  B  describes  the  three 
types  of  IPTs.  These  cross-functional  teams  are  formed  to 
deliver  a  product  with  common  performance  objectives  using  a 
common  approach  for  which  they  hold  themselves  mutually 
accountable.  Members  of  an  IPT  represent  the  technical, 
manufacturing,  business,  and  support  organizations  that  are 
critical  to  developing,  procuring,  and  supporting  the 
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product.  Team  members  work  together  to  achieve  the  team's 
objectives . 

E.  LOGISTICIANS  AND  INTEGRATED  PRODUCT  TEAMS 

As  functional  area  experts  logisticians  have  special 
responsibilities  because  they  bring  special  knowledge  and  a 
special  point  of  view  to  the  effort.  The  degree  to  which 
these  experts  are  willing  to  share  their  knowledge  and  point 
of  view  will  determine  their  value  to  the  team.  In  addition 
to  providing  an  expert  opinion,  experts  play  an  important 
training  role  on  the  team.  By  sharing  their  expertise,  they 
educate  fellow  team  members  to  the  not-so-obvious 
implications  of  programmatic  decisions  and  actions.  Team 
member  involvement  includes  active  participation,  effective 
communication,  challenging  requirements  that  do  not  make 
sense,  and  paying  attention  to  detail.  For  the  logistician, 
dedication  to  these  principles  can  make  the  difference 
between  fielding  a  costly,  ineffective,  inefficient  system 
or  an  optimal  one.  Optimal  solutions  are  often  a  result  of 
well-researched  opinions,  constructive  conflict  resolution, 
and  tenacity. 

'  Because  with  few  exceptions  most  of  the  cost  of  a 
program  is  in  the  cost  of  ownership,  i.e.  the  support  of  the 
system  throughout  its  operational  life,  the  logistician  can 
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make  major  contributions  to  the  acquisition  of  a  cost- 
effective  system.  While  dealinq  with  short-term  problems, 
the  logistician  must  also  think  about  problems  that  will 
arise  in  the  future.  For  example,  increased  environmental 
awareness  and  legislation  has  increased  the  difficulty  and 
cost  of  demilitarization  and  disposal  of  systems.  An 
unreliable  system  with  poor  maintenance  support  will  drive 
the  need  to  procure  costly  spare  components  that  may  or  may 
not  be  available.  Identification  of  such  problems  early  in 
the  concept  exploration  phase  of  a  program  can  help  avoid 
serious  consequences  later  in  the  program's  life  cycle. 

The  logistician' s  role  on  an  IPT  depends  on  the  type  of  IPT 
it  is.  On  a  higher  level  IPT  not  directly  focused  on 
support  issues  the  logistician  should  be  concerned  with 
identifying  and  highlighting  the  long-term  logistics 
implications  of  various  program  issues.  He  may  then  form  a 
supportability  IPT  to  focus  on  mitigating  the  effects  of 
those  issues  on  the  supportability  of  the  system.  At  the 
program  level  the  logistician  should  be  more  concerned  with 
influencing  the  design  of  the  system  and  the  design  of  the 
support  structure.  An  important  responsibility  of  the 
logistician  on  an  IPT  is  to  help  the  team  create 
supportability  performance  requirements  that  are 
quantifiable  and  testable  so  that  the  decision-makers  can 
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gain  insight  into  the  operational  suitability  of  the  product 
and  the  logistic  planners  can  plan  for  the  support  of  the 
item. 

The  Fleet  Support  Team  (FST)  is  an  example  of  an  IPT 
directed  to  interface  with  the  fleet,  identify  problems,  and 
develop  solutions.  The  focus  of  these  teams  is  on  all 
aspects  of  life  cycle  support  for  a  system  from  when  it  is 
fielded  until  it  is  disposed.  Within  the  context  of 
Affordable  Readiness,  the  FST  is  the  center  for  identifying 
initiatives,  performing  analyses,  and  developing  action  - 
plans.  The  resulting  action  plans  can  include  investments 
(Engineering  Change  Proposals) ,  process  improvements 
(consolidated  maintenance  activities) ,  or  innovative  support 
solutions  (commercial  or  joint  service  support  equipment) . 
IPTs,  including  FSTs,  are  not  standardized.  Each  PMA 
determines  how  he  will  manage  his  program.  Some  teams  are 
highly  specialized  while  others  cross  diverse  functional 
barriers.  Some  FSTs  are  organized  within  the  program  office 
while  others  are  organized  within  the  aircraft  controlling 
custodian  or  depot  maintenance  engineering  support 
organizations.  Accordingly,  the  APML  has  different 
relationships  with  teams,  both  within  and  beyond  the  program 
office,  as  determined  by  individual  program  office 
organization.  Additionally,  program  offices  use  a  number  of 
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different  sources  for  logistics  support.  Some  program 
managers  rely  heavily  on  Navy  personnel  assigned  within  the 
program  office,  others  rely  on  government  employees  from  a 
logistics  competency  group  in  NAVAIR,^  while  some  contract 
out  to  commercial  sources  for  logistics  analyses  and 
recommendations . 

F.  CHANGE  FROM  CHECKLISTS  TO  INTEGRATED 

FLEXIBILITY 

Because  acquisition  program  management  is  tailored  to 
meet  individual  program  needs,  the  challenge  of  supporting 
information  systems  is  to  gather  and  present  data  in  an 
equally  flexible  manner.  The  data  must  reflect  performance 
and  supportability  metrics  in  a  meaningful,  effective,  and 
efficient  way. 


5  Within  NAVAIR's  matrix  organization  in  which  team  members 
report  to  both  a  functional  leader  and  a  program  leader, 
competency  leaders  are  responsible  for  providing  skilled, 
knowledgeable  members  for  IPTs  and  for  managing  the 
processes  by  which  these  personnel  support  the  teams.  The 
Logistics  Competency  Center's  mission  is  to  provide  the 
people,  skills,  knowledge,  processes,  facilities,  and 
equipment  required  to  manage  and  perform  the  planning, 
development,  acquisition,  integration,  and  delivery  of  all 
integrated  logistic  support  elements  necessary  to  affordably 
design,  support  and  maintain  weapons  systems  throughout  the 
program's  life-cycle.  Supporting  missions  include  technical 
publication  development,  logistics  elements  integration, 
affordability  studies,  and  engineering  technical  services  to 
name  but  a  few. 
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Traditional  Integrated  Logistics  Support  (ILS)  used  a 
step-by-step  analytical  process  that  defined  all  the 
logistics  and  maintenance  tasks,  resources,  and  requirements 
necessary  to  establish  and  sustain  an  effective  support 
program  over  the  life  cycle  of  a  program.  The  logistics 
community  depended  on  ILS  products  generated  from  the 
application  of  the  Logistics  Support  Analysis  (LSA)  process 
as  stipulated  in  MIL-STD-1388-1A.  Now  as  DoD  5000. 2-R 
stipulates,  supportability  factors  are  to  be  integrated 
elements  of  program  performance  specifications,  but  support 
requirements  are  not  to  be  stated  as  distinct  logistics 
elements.  Instead  they  are  to  be  stated  as  performance 
requirements  that  relate  to  a  system' s  operational 
effectiveness,  supportability,  and  life  cycle  cost 
reduction.  The  challenge  to  the  PMA  is  to  develop  a 
performance-based  statement  of  work  that  includes 
supportability  metrics  in  addition  to  the  usual 
operationally  oriented  performance  goals .  Programs  must  be 
able  to  be  evaluated  on  specific  performance  metrics 
describing  the  relation  of  cost-of-ownership  with  other 
parameters  such  as  mission  performance,  safety  and 
availability/readiness.  Appendix  C  describes  four  factors 
that  must  be  considered  in  establishing  supportability 
requirements. 
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MIL-HDBK-502  offers  guidance  on  acquisition  logistics 
as  an  integral  part  of  the  systems  engineering  process.  The 
information  is  applicable,  in  part  or  in  whole,  to  all  types 
of  material  and  automated  information  systems  and  all 
acquisition  strategies  but  it  is  not  a  "cookbook"  approach 
to  acquisition  logistics.  It  is  intended  to  accommodate  the 
vast,  widely  varying,  array  of  potential  material 
acquisitions  and  provides  general  guidance  to  logisticians 
on  how  to  perform  certain  aspects  of  their  jobs. 

NAVAIR  has  a  companion  "Contracting  for  Supportability 
Guide."  This  guide  identifies  five  steps  (Appendix  D)  to  be 
used  to  establish  a  supportability  strategy  for  acquisition 
programs  for  new  systems,  major  and  minor  modifications  or 
upgrades,  and  commercial  and  non-developmental  items. 
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III.  LOGISTICS  MANAGEMENT  DECISION 

SUPPORT  SYSTEM 


A.  HISTORY  2^  ORIGINAL  REQUIREMENTS 

The  LMDSS  requirement  evolved  from  NAVAIR  and  Aviation 
Supply  Office  (ASO)  initiatives  to  assess  the  logistics 
health  of  programs  using  standard  metrics  of  readiness, 
supportability,  and  cost.  In  1991,  the  effort  was  expanded 
into  a  requirement  to  develop  a  decision  support  system 
which  would  be  the  primary  tool  for  APML/Weapon  System 
Managers  (WSMs)  to  achieve  a  "continuous,  measurable 
reduction  in  life~cycle  costs  while  maintaining  operational 
readiness  and  sustainability."  (LMDSS  Req.  Doc.,  1993) 

The  driving  requirement  behind  LMDSS  was  the  need  for  a 
tool  to  facilitate  measurement  to  plan  and  identification  of 
targets  for  cost  reduction.  This  would  be  accomplished 
through  use  of  a  software  package  operating  on  several  key 
databases  organized  in  a  relational  environment.  The  key  to 
successful  operation  of  the  DSS  rested  with  the  integration 
of  diverse  databases  into  a  central  repository.  This 
integration  would  result  in  an  immediate,  precise  response 
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to  queries,  regardless  of  the  type  data  requested  or  its 
origin. 

Access  to  logistics  data  in  a  relational  environment 
allowed  a  level  of  analysis  that  was  impossible  in  personal 
computer  based  systems.  Where  analytical  requirements 
involved  databases  outside  the  repository,  access  would  be 
made  automatically  as  a  standard  function  of  LMDSS.  The 
repository  would  encompass  data  from  all  three  maintenance 
levels,  the  supply  community  and  selected  sources  of 
specialized  information.  To  speed  access,  the  system  was  to 
be  established  using  both  local  and  wide  area 
interconnection  techniques.  The  system  was  to  be  designed 
to  accommodate  managers  and  analysts  who  were  not 
necessarily  Automated  Data  Processing  (ADP)  experts. 

The  NAVAIR  development  team  determined  that  to  provide 
maximum  utility,  LMDSS  needed  to  provide  both  a  structured, 
modular  approach  and  an  unlimited  ad  hoc  query  capability. 

A  requirement  for  an  extensive  repertoire  of  Statistical 
Process  Control  and  traditional  charting  available  on  a 
semi-automated  basis  was  established  to  compliment  both  of 
these  capabilities. 

The  plan  was  for  the  data  repository  to  be  located  at 
ASO  to  provide  all  Naval  Aviation  maintenance  personnel, 
engineering  personnel,  supply  personnel  and  procurement 
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personnel  a  "common  and  integrated  sheet  of  music."  (LMDSS 
Req.  Doc.,  1993).  Provisions  would  be  made  to  integrate 
Streamline  Alternative  Logistics  Transmission  System  (SALTS) 
data  with  traditional  Aviation  Maintenance  and  Material 
Management  (AV3M)  reporting  when  available.  The  objective 
was  to  incorporate  all  necessary  AV3M  and  Uniform  Inventory 
Control  Point  (UICP)  data,  so  that  it  could  be  manipulated 
directly  by  the  logistician,  to  support  detailed  causative 
research.  The  NAVAIR  development  team  also  desired  summary 
data,  where  refined  and  available.  However,  the  focus  was 
to  remain  on  ability  to  support  detailed  research  in  a 
relational  database  management  system  environment. 

The  LMDSS  software  was  to  be  divided  logically  into 
five  modules  of  1)  Candidate  Selection;  2)  Problem 
Isolation;  3)  Ad  hoc  Queries  and  Special  Summaries;  4) 
Evaluation  and  Selection  of  Alternatives  and  5) 
Implementation  and  Status  Tracking. 

The  original  requirements  for  LMDSS  assumed  a  IBM  RISC 
System/6000  Model  970  machine  located  at  ASO  hosting  a 
massive  -  hundreds  of  gigabytes  -  database  and  regional  IBM 
RISC  System/6000  machines  located  at  each  Naval  Aviation 
Depot  (NADEP)  and  selected  Naval  Air  Warfare  Center  (NAWC) 
activities.  The  regional  machines  would  not  house  the 
extensive  data  held  in  the  NALDA  ASO  IBM  RISC  System/6000 


19 


machine.  The  regional  machines  would  tap  the  ASO  database 
resource  via  remote  procedure  calls.  Connectivity  between 
the  RISC  machines  would  be  provided  by  either  direct 
connection  of  each  RISC  system  to  the  NAVNET  Internet  or 
directly  to  the  DON  MILNET.  It  would  employ  a  client/server 
distributed  architecture  linking  these  computers  via  TCP/IP 
in  a  WAN  and  LAN  environment.  ASO  LMDSS  customers  would 
directly  interact  and  conduct  X  Windows  client/server 
sessions  with  the  ASO  RISC  machine  via  LAN  and  TCP/IP.  All 
other  LMDSS  customers  would  directly  interact  and  conduct  X 
Windows  client/server  sessions  with  their  respective 
regional  RISC  system  via  LAN  and  TCP/IP.  It  was  planned  as 
an  information  system  requiring  direct  LAN  connectivity. 

The  original  system  was  not  planned  to  support  PCs  or 
workstations  in  stand-alone  mode,  nor  support  connectivity 
via  modem. 

All  programming  was  to  be  done  in  Ada.  The  operating 
system  for  the  RISC  machines  was  to  be  IBM  AIX  and  Oracle 
was  to  be  the  database.  Rapid  prototyping  was  used  for  the 
software  development  methodology.  The  prototype  system  is 
running  on  a  RISC  machine  located  at  ASO. 

By  1995,  it  was  determined  that  Ada  was  unsuitable  as  a 
host  language.  HTML  and  PERL  were  used  to  request  and 
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display  reports.  X  Windows  and  C  were  being  used  for 
prograraming  complex  displays. 

In  1996,  the  scope  of  the  LMDSS  underwent  substantial 
change.  The  LMDSS  database  was  expanded  to  essentially 
become  NALDA  Phase  II  (NAVAIR  3.6.2,  1998).  The  LMDSS  did 
not  start  out  as  the  heart  of  NALDA  II  but  at  that  point, 
that  is  what  the  system  became.  Appendix  E  provides  a 
detailed  discussion  of  the  evolution  of  NALDA  and  the 
composition  of  NALDA  Phase  II. 

In  1997,  due  to  contractor  difficulties  and  equipment 
problems,  the  development  team  decided  to  convert  reports  to 
operate  with  commercial  browsers,  abandon  X  Windows  and  use 
Active  X  controls  for  complex  reports,  and  move  to  Internet 
access.  Additionally,  they  decided  to  transition  from  the 
IBM  RISC/6000  machines  to  multinode  IBM  Scaleable 
P0WERparallel2  (SP2)  for  the  production  platform.  The  SP2s 
will  all  be  located  at  NAS  Patuxent  River  Maryland,  rather 
than  the  distributed  architecture  originally  planned. 
Expectations  for  what  the  system  would  ultimately  deliver 
were  scaled  back.  Rather  than -being  everything  for 
everyone,  core  capabilities  were  identified,  and  "nice  to 
haves"  were  eliminated.  Those  capabilities  eliminated 
included  graphic  capabilities,  the  Return  on  Investment 
module,  and  ad  hoc  query  access  against  the  detail  data. 
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The  ad  hoc  query  capability  would  still  be  possible  using 
the  IQ  Tool  -  an  OLAP  software  product.  Because  of  the 
detailed  knowledge  of  the  database  and  SQL  skills  required 
to  make  effective  use  of  this  software,  the  tool  would  be 
available  only  to  certain  users.  These  users  would 
primarily  be  analysts  at  the  major  claimants  or  the  NADEPs. 

B.  THE  CURRENT  STATE  OF  THE  LMDSS 

The  data  load  into  the  LMDSS/NALDA  II  database  on  the 
SP2  equipment  is  now  in  its  final  stages.  When  this  load  is 
complete  the  application  will  be  pointed  to  the  new  tables 
and  turned  over  to  the  Fleet  as  the  replacement  for  NALDA 
Phase  I.  Automated  database  loading  software  is  operational 
and  AV3M  SALTS  submissions  are  now  being  loaded  directly 
into  the  LMDSS/NALDA  II  database.  Naval  Sea  logistics 
Command  (NSLC)  is  now  out  of  the  AV3M  business  (NAVAIR 
3.6.2,  1997) . 

C.  THE  LMDSS  FEATURES  AND  CAPABILITIES 

The  LMDSS  is  organized  into  seven  functional  areas. 

The  following  descriptions  of  these  areas  and  subareas  have 
been  derived  from  the  LMDSS  homepage: 
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1.  Management  Analysis 

This  module  consists  of  various  tools  for  displaying 
high  level  summary  data  for  end  items,  claimants  or' 
organizations.  These  tools  identify  system  degraders  and 
produce  reports  ranked  by  parameter.  The  reports  include: 

•  End-Item  Matrix.  This  produces  a  matrix  that 
summarizes  reliability,  supportability  and  cost  data 
to  the  end-item  level. 

•  Claimant  Matrix.  This  produces  a  matrix  that 
summarizes  reliability,  supportability  and  cost  data 
to  the  claimant  level. 

•  End-Item/Claiment  Matrix.  This  produces  a  matrix 
that  summarizes  reliability,  supportability  and  cost 
data  to  the  end-item  and  claimant  level. 

'  *  Organization  Matrix.  This  produces  a  matrix  that 

summarizes  reliability,  supportability  and  cost  data 
to  the  organization  level. 

•  Beyond  Capability  Maintenance  (BCM)  Report.  This 
produces  a  report  that  summarizes  BCM  data  to  the 
organization  level. 

2.  Candidate  Identification 

This  module  consists  of  various  tools  for  displaying 
reliability,  supportability,  and  cost  summary  parameters  for 
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selected  aviation  equipment.  These  tools  identify  system 
degraders  and  produce  reports  ranked  by  parameter.  The 
primary  purpose  of  this  module  is  to  allow  managers  to 
identify  opportunities  for  life  cycle  cost  and  readiness 
improvement.  These  tools  include: 

•  Reliability/Supportability/Cost  (R/S/C)  Matrix. 

This  offers  a  choice  of  three  basic  matrices: 
Component  by  Reported  NIIN,  NUN  Head  of  Family,  and 
Work  Unit  Code  (WUC) . 

•  Common  Equipment  Matrix.  This  identifies  potential 
problems  with  cross-platform  components. 

Additionally,  there  are  four  methods  for  collection 
of  equipment/systems/components  that  have  multiple 
aircraft  applicability: 

■  Local  Routing  Code  (LRC) .  This  uses  the  ASO  local 
routing  code.  This  selection  will  only  collect 
data  for  NIIN  or  NIIN  Heads  of  family  that  match 
the  specified  LRC. 

■  Type  Model.  This  collection  method  is  based  on 
the  minimum  number  of  Type  Models  in  which  a  NIIN 
must  occur  to  be  considered  common  equipment. 

■  Type  Model  Series  (TMS) .  This  method  is  based  on 
the  minimum  number  of  TMSs  in  which  a  NIIN  must 
occur  to  be  considered  common  equipment. 
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■  NIIN.  This  selection  displays  the  selected  NIIN, 
its  nomenclature,  the  Type  Equipment  Code 
(TEC) /TMS  it  can  be  found  on,  and  the  number  of 
the  selected  items  in  the  TEC/TMS. 

•  Emergent  Problems  Matrix.  This  identifies  items 
that  show  recent  changes  in  reliability,  cost, 
maintenance  and  supply., 

•  Bureau  Number  Matrix.  Displays  support  costs, 
maintenance  and  supply  statistics,  and  reliability 
figures  broken  out  by  individual  bureau  number  for  a 
TMS. 

3.  Trend  Analysis 

This  module  consists  of  tools  used  to  analyze  system 
degraders  to  determine  the  basic  problem (s)  and  examine  the 
underlying  cause (s) .  The  Historical  Trend  Analysis  tools 
display  reports  of  statistics  over  time.  This  is  tabular 
information  summarized  by  month  covering  End  Item  or  the 
component  levels. 

•  Aircraft  Utilization  History.  Presents  a  parameter 
value  entry  and  selection  form  to  prepare  for  the 
display  of  flight  and  utilization  data  that  aids  in 
historical  trend  analysis  of  aircraft. 
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•  Work  Unit  Code  History.  Presents  a  parameter  value 
entry  and  selection  form  that  includes  selection  of 
WUCs,  to  prepare  for  the  display  of  maintenance  data 
that  aids  in  historical  trend  analysis  of  WUCs. 

•  Intermediate  Maintenance  Activity.  Presents  a 
parameter  value  entry  and  selection  form  which 
includes  input  of  NUN  and  Date  Range  to  prepare  for 
the  display  of  intermediate  level  maintenance  that 
aids  in  trend  analysis. 

4.  Cost  Analysis 

This  module  contains  tools  used  to  analyze  end-item  and 
component  cost  data. 

•  Annual  Operations  and  Support  Costs  (AIR  4.2). 
Displays  platform  level  cost  information  based  on 

'  the  OPNAV  Code  N889  sponsored  Navy  Flying  Hour 

Program.  The  report  can  be  generated  for  a  specific 
TMS  squadron  manned  at  90%  or  100%. 

•  Budget  Analysis  (OP-20  Report) .  Displays  the  Budget 
Analysis  Report  selection  parameters.  The  operator 
selects  the  fiscal  year,  TEC  and  funding  command  to 
produce  the  desired  report. 
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•  Labor  Cost  History.  Displays  Labor  Cost  History 
data  in  tabular  format.  Parameters  of  year  and 
maintenance  level  may  be  selected. 

•  Inflation  Factors.  Displays  fiscal  year,  inflation 
rate,  raw  index,  weighted  index  and  budget  year 
multiplier  for  a  variety  of  operator  selectable 
appropriations  categories. 

•  Item  Value  Cost  Reports.  Allows  users  to  select 
between  the  Item  Value  to  Depot  Repair  Cost  report 
or  the  Item  Value  to  Labor  Cost  report.  In  the  Item 
Value  to  Depot  Repair  Cost  report  the  user  can 
compare  the  replacement  value  of  items  in  the 
database  to  the  cost  of  level  3  maintenance  for  a 
selected  time  period.  The  result  is  shown  as  a 
percentage  showing  the  level  3  maintenance  cost  of 
in  service  units  compared  to  the  cost  to  replace 
those  units.  The  Item  Value  to  Labor  Cost  reports 
are  similar  in  that  they  compare  the  replacement 
cost  of  items  to  the  labor  cost  at  maintenance 
levels  1  and  2  for  those  items.  The  results  show 
the  annual  cost  of  MLl  and  ML2  labor  as  a  percentage 
of  item  replacement  cost. 
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5.  Detailed  Analysis 

This  module  contains  tools  used  to  analyze  items 
identified  in  the  Candidate  Identification  Function. 

•  Detailed  Component  Report  (DCR) .  This  is  a 
comprehensive  report  encompassing  data  from  all 
three  levels  of  maintenance  (AV3M)  and  supply 
(Weapon  System  File/UICP) .  It  is  designed  to  fault 
isolate  candidates  from  the  candidate  identification 
to  the  root  hardware  cause (s)  of  R/S/C  degradation. 
This  is  accomplished  through  drilling  down  through 
progressively  more  specific  forms  to  lower  levels  of 
detail . 

•  Supply  Synopsis.  This  section  of  the  Detailed 
Component  Report  provides  greatly  expanded 
information  covering  supply  and  depot  repair 
parameters . 

•  Source  Document  Report  (VIDS/MAF) .  This  section 
provides  detailed  report  information  of  VIDS/MAFs. 

It  is  possible  to  drill  down  to  and  view  a  specific 
VIDS/MAF. 

6 .  Supply  Analysis 

This  module  consists  of  specialized  summary  and 
forecasting  reports  intended  for  use  by  supply  personnel. 
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This  utility  provides  a  means  through  which  both  readiness 
and  cost  factors  are  examined  concurrently. 

•  Wholesale  System  Demand.  This  form  is  used  to  aid 
in  forecasting  demand  for  specific  NIINs.  It  will 
link  to  a  NIIN  Analysis  screen  that  will  provide  the 
user  with  a  breakdown  of  more  NIIN  specific 
information  and  links  to  the  DCR  and  Tools  Cross- 
Reference  report. 

•  Wholesale  System  Investment.  This  report  is  sorted 
by  NIIN  and  a  break  out  of  repair  costs  is 
displayed. 

•  End-Item  Material  Issue  Trends.  This  report  can  be 
displayed  by  TEC  or  TMS. 

•  Average  Customer  Wait  Time  Reports.  These  reports 
can  be  produced  for  specific  TECs  broken  out  by 
maintenance  level,  response  time  crossed  with  COG  or 
the  highest  wait  days  across  all  NIINs  for  that  TEC. 

•  Wait  Time  Maintenance  Impact.  Under  Development 

•  Average  Days  to  Receipt.  Under  Development 

•  Backorder  History  Report.  Allows  for  the  display  of 
data  for  TEC/TMS  sorted  by  NAVICP  Material  Type  and 
Data  Elements.  The  report  may  also  include  DLA 
Supply  Center  and  Data  Elements. 
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•  Backorder  Ranked  Report.  This  report  allows  for 
identification  of  the  NIINs  with  the  largest  number 
of  backorders  against  them. 

•  Planned  vs  Actual  Opportunity  Cost.  This  form 
allows  the  user  to  enter  source  of  reliability  data 
to  report  on  from  NAVICP,  LSAR,  or  Manually  Entered. 

•  Mean  Flight  Hours  Between  Failures  Report.  This 
form  and  resulting  report  can  be  used  for  "what  if" 
analysis.  Planned  data  can  be  entered  and  then 
compared  to  actual  data. 

•  NAVICP  NSN  Snapshot.  This  report  may  be  generated 
for  a  specific  NUN.  Either  a  summary  report  or  a 
report  specific  to  backorders,  stock  status, 
alternate  NUN,  etc.  may  be  produced. 

•  Repair  Cycle  Report.  Gathers  and  displays  data  on 
repair  cycle  and  BCM  rates. 

7 .  Engine  Analysis 

This  module  consists  of  tools  that  allow  the  analyst  to 
view  projected  actual  costs  and  hours  for  different  engines. 

•  Depot  Engine  Repair  Cost.  This  provides  the  analyst 
with  historical  information  on  the  cost,  funding 
source,  and  activity  for  each  engine.  This  report 
is  not  accessible  to  contractors. 
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•  Engine  Overview.  The  overview  report  expands  the 
information  available  to  include  engine  inventory,  a 
cost  breakout  and  the  capability  to  select  data 
covering  all  TMSs  for  specific  TMSs,  all  sites,  PAC 
sites  or  LANT  sites. 

•  Engine  Removal  Analysis.  Tracks  average  engine  time 
since  last  repair  when  removed.  The  repair  site  and 
the  number  of  engines  attributed  to  that  specific 
site  are  also  listed. 

•  Engine  Removal  Trend.  Charts  the  engine  removals'  to 
TMS  based  on  100-hour  service  intervals  since  last 
repair.  Multiple  series  may  be  compared  on  the  same 
chart,  giving  insight  into  factors  such  as  "infant 
mortality"  and  "high  time." 

•  Top  Reasons  for  Removal.  Displays  the  top  reasons 
for  engine  removals  to  TMS. 

•  Flight  Hours  Since  Last  Repair.  This  is  a  companion 
report  to  the  Engine  Removal  Trend  Report .  In 
addition  to  charting  removals  by  TMS,  it  also  shows 
the  reason  for  each  removal  along  with  flight  hours 
in  100-hour  increments. 

•  Flight  Hours  Since  Repair  at  Removal.  Displays 
engine  removals  and  maintenance  man-hours.  This 
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report  differentiates  between  scheduled  and 
unscheduled  maintenance,  and  displays  average  hours. 

•  Engine  Demand  Forecasting.  Based  on  the  premise 
that  evolving  reliability  and  maintainability  data, 
both  historically  derived  and  imposed,  must  be 
considered  in  conjunction  with  historical  demand  for 
successful  engine  demand  forecasting.  This  option 
derives  from  historical  files  and  extrapolates  past 
performance  to  future  needs  through  application  of 
relative  flying  hours. 

8.  Reference  Information 

This  module  consists  of  tools  that  provide  general 
aircraft  information,  definitions,  statistics,  assistance, 
reference  information/reports,  and  information  about  the 
application/database . 

9 .  implication  Management  Tools 

Contains  various  utilities  that  provide  current 
database  status,  user  identification,  server  status,  and  the 
Software  Change  Request  form. 

10 .  Feature  Synopsis 

This  provides  a  brief  description  of  the  modules  and 
links  to  the  sub-elements. 
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The  LMDSS  has  an  on-line  Help  capability.  Each  input 
form  has  an  accompanying  Help  function  that  explains  the 
input  fields.  Additional  links  within  some  of  the  Help 
screens  provide  information  on  algorithms  used  to  derive  the 
reports.  Other  links  provide  definitions  of  acronyms  and 
computer  jargon. 

D.  QUALITY  ASSURANCE  (QA)  PLAN 

Prior  to  the  transition  from  the  NALDA  I  to  NALDA  II 
database,  a  systematic  series  of  tests  to  ensure  proper 
functionality,  accuracy  and  a  smooth  transition  is  planned. 
The  two  primary  targets  of  this  testing  are  to  assure 
accurate  data  retrieval  throughout  the  application  and 
database  integrity. 

Sections  of  the  application  and  certain  derived  data  in 
the  Oracle  tables  where  problems  have  been  previously 
identified  are  being  re-coded.  Concurrent  with  this  effort, 
the  QA  team,  composed  of  analysts  conversant  with  LMDSS  and 
NALDA  I  will  go  through  each  section  of  the  LMDSS 
application  in  a  critical  review  of  form,  format,  and 
accuracy  of  content. 

The  prototype  application  software  currently  in  use  is 
identical  to  that  which  will  operate  against  the  SP2s.  This 
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means  that  quality  assessment  can  begin  immediately.  There 
are  specific  challenges  that  must  be  addressed: 

1 .  Data  Ou'^uts  are  not  Identical 

Although  the  database  which  LMDSS  now  reaches  is 
based  on  a  data  pull  from  NALDA  I,  the  data  outputs  are  not 
identical.  This  is  because  in  LMDSS,  the  data  pre¬ 
processing  has  been  improved  to  provide  greater  accuracy. 
Examples  of  the  differences  include  use  of  aircraft  versions 
in  addition  to  TMS  and  revised  item  count  logic.  This  makes 
comparative  analysis  difficult,  but  not  impossible. 

2 .  Software  Change  Requests  System 

The  problem  is  that  the  reporting  system,  which  was 
built  into  the  client-server  version  of  LMDSS,  became 
inactive  during  the  transition  to  the  Internet  based 
version.  It  has  been  two  years  since  a  Software  Change 
Request  (SCR)  has  been  classified,  scheduled  or  answered. 
This  has  resulted  in  a  lack  of  systematic  documentation  of 
problems  and  resolutions  during  the  transition  (Jones, 

1998). 

The  SCR  system  is  again  working.  All  SCRs  and  their 
status  can  be  viewed  under  the  Application  Management  Tools 
Module  of  the  LMDSS  application.  An  examination  of  the 
current  list  of  SCRs  indicates  that  there  has  been 
significant  progress  made  on  the  application  validation. 
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E.  ANTICIPATED  ADVANTAGES 

The  development  team  anticipates  that  the  LMDSS  will 
provide  an  important  analysis  function  to  assist  logistics 
managers  in  establishing  requirements  for  new  acquisitions 
as  well  as  troubleshooting  existing  weapon  systems.  Because 
of  the  Integrated  Data  Environment  (IDE) ,  the.  LMDSS  will 
allow  all  potential  user  levels  to  substantially  reduce  the 
amount  of  time  required  to  identify  and  analyze  problems  in 
logistics  support. 

The  LMDSS  will  provide  an  ability  to  analyze  data 
concerning  common  equipment  -  those  items  that  are  used  on 
more  than  one  weapons  platform.  The  present  approach 
essentially  looks  at  present  conditions  and  backordering 
philosophies  that  encourage  a  tunnel  Vision  approach  due  to 
difficulty  in  identifying  common  components  that  can  be  used 
across  platforms/weapon  systems  in  correlating 
maintenance/repair  and  ordering  of  specific  common 
equipment . 

With  the  implementation  of  the  LMDSS,  daily  feeds  of 
maintenance  and  flight  data  will  be  received  through  SALTS 
terminals  directly  to  the  production  database.  The  current 
system  uses  a  monthly  update  cycle  based  upon  the  NSLC 
processing  schedule  that  establishes  data  as  45  days  old  as 
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the  best  case.  Monthly  reports  will  be  available  at  least 
30  days  earlier  under  the  new  system.  Data  quality  will 
also  be  improved  with  the  strengthening  of  validation 
specifications  at  the  source  and  use  of  "most  probable 
logic"  within  the  data  summarization  function  of  the 
database.  (NAVAIR  7.0,  1997) 
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IV.  LITERATURE  REVIEW 


A.  THE  DECISION  SUPPORT  SYSTEM  AREAS  OF  RESEARCH 

Much  of  this  DSS  research  can  be  classified  into  one  of 
five  areas: 

1)  what  distinguishes  the  DSS  from  other  computer 
information  technologies; 

2)  whether  the  DSS  actually  improves  decision  quality 
or  performance; 

3)  identifying  specific  design  characteristics  and  the 
impact  they  have  on  the  DSS  development; 

4)  the  role  of  the  decision-maker  and  how  differences 
between  individuals  and  organizations  can  influence 
the  effectiveness  of  the  DSS;  and 

5)  DSS  evaluation  methods. 

1.  The  DSS  Versus  Other  Ccnnputer  Information 
Technologies 

The  first  area  of  research  has  addressed  what 
distinguishes  the  DSS  from  other  computer  information 
technologies.  Currently,  there  is  a  general  consensus  that 
a  DSS  is  composed  of  the  following  three  interrelated 
components:  data  management,  model  management,  and  dialogue 
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management  components  (Alter,  1977;  Sprague,  1980;  Keen, 
1981) .  Each  component  provides  specific  capabilities  to  a 
decision-maker  and  improves  the  effectiveness  with  which  he 
or  she  works. 

The  data  management  component  should  include: 

1)  the  capture  and  extraction  of  data  into  a  database; 

2)  the  storage,  retrieval,  and  control  of  data  by  a 
database  management  system; 

3)  the  ability  to  interact  with  data  from  internal  and 
external  sources; 

4)  the  ability  to  perform  ad  hoc  queries;  and 

5)  the  flexibility  to  allow  rapid  additions  and 
changes  in  response  to  unanticipated  user  requests. 
(Alter,  1977;  Sprague,  1980;  Keen,  1981) 

The  model  management  component  of  DSS  provides  a  user 
with  a  set  of  capabilities  that  differentiate  it  from  other 
traditional  computer  systems.  These  capabilities  include: 

1)  the  use  of  multiple  models  to  support  diverse 
problems; 

2)  the  support  of  semi-structured  and  unstructured 
problems; 

'  3)  the  ability  to  build  models  quickly  and  easily; 

4)  the  ability  to  track  models  through  a  model 
directory; 
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5)  the  ability  to  integrate  models  with  appropriate 
links  through  the  database;  and 

6)  the  creation,  retrieval  and  storage  of  models 
handled  by  a  model  base  with  management  functions 
analogous  to  database  management.  (Alter,  1977; 
Sprague,  1980;  Keen,  1981) 

The  dialogue  management  component  provides  the  mode  of 
interaction  between  the  user  and  the  DSS.  Research  results 
suggest  that  a  well-developed  interface  should  include: 

1)  the  support  of  multiple  dialog  styles; 

2)  the  capture,  storage,  and  analysis  of  dialogue 
through  a  dialog  management  system; 

3)  the  ability  to  interact  with  the  model  and  data 
components  of  the  DSS;  and 

4)  the  support  of  multiple  methods  of  presenting 
output  to  provide  for  a  variety  of  formats  and 
media,  and  the  flexibility  for  different  users' 
knowledge  base.  (Alter,  1977;  Sprague,  1980;  Keen, 
1981) 

2.  DSS  Benefits 

The  second  group  of  studies  has  primarily  focused  on 
benefits  that  the  DSS  provide.  The  primary  justification 
for  the  development  of  a  DSS  is  that  it  will  be  a  value  to 
the  decision  maker  (Hogue  and  Watson,  1983) .  Some  studies 
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in  this  area  support  the  premise  that  the  DSS  improves 
decision  quality  or  effectiveness  (Sprague  and  Watson,  1986; 
Hogue  and  Watson,  1985;  Sprague  and  Carlson,  1982;  Keen  and 
Scott  Morton,  1978).  In  other  studies  there  was  no  effect, 
and  in  still  others  decision  quality  worsened  when  a  DSS  was 
employed  (Benbasat  and  Nault,  1990;  Sharda,  et  al.,  1988).  A 
DSS  provides  a  coherent  strategy  for  going  beyond  the 
traditional  use  of  computers  in  structured  situations  where 
measures  of  effectiveness  and  efficiency  are  nearly 
equivalent.^  In  the  semi-structured  and  unstructured 
situations  in  which  the  DSS  is  used,,  effectiveness  has  been 
the  primary  focus.  Other  research  suggests  that  it  is 
efficiency  that  is  actually  improved  (Todd  and  Benbasat, 
1992)  . 

The  DSS  is  designed  to  be  an  interactive  system  used  by 
managers  with  little  experience  in  computers  and  analytical 
methods  to  help  improve  the  effectiveness  and  productivity 
by  supporting,  rather  than  replacing,  judgment  (Fedorowicz 
and  Manheim,  1986) .  A  DSS  is,  in  effect,  an  assistant  to 
whom  the  manager  delegates  activities  involving  retrieval, 

®  Efficiency  is  performing  a  given  task  as  well  as  possible 
in  relation  to  some  predefined  performance  criterion.  It  is 
a  measure  of  resources  utilized  against  results  derived. 
Effectiveness  involves  identifying  what  should  be  done  and 
ensuring  that  the  chosen  criterion  is  relevant.  It  is  the 
degree  to  which  a  goal  is  achieved. 
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computation,  reporting,  and  development  of  alternatives.  A 
manager  evaluates  the  results  and  chooses  the  next  step. 

The  benefits  the  DSS  provides  are  often  not  quantitative. 
Benefits  include  the  ability  to  examine  more  alternatives, 
stimulate  new  ideas,  improve  confidence  in  the  decisions, 
reduce  the  probability  of  error,  improve  communication  of 
analysis,  and  speed  up  decision  making.  (Keen  and  Morton, 
1978;  Keen,  1986;  Hogue  and  Watson,  1985;  Hogue  and  Watson, 
1983) 

3 .  Design  Characteristics 

The  third  area  of  DSS  research  has  been  directed 
towards  identifying  specific  design  characteristics  and  the 
impact  they  have  on  the  DSS  development.  Topics 
investigated  have  included  the  impact  of  different 
presentation  formats,  the  use  of  color,  the  influence  of 
different  graphics  capabilities,  and  the  influence  of 
different  user  interfaces.  Analysis  has  generally  provided 
mixed  results  as  to  the  impact  of  these  factors  on  decision 
making  effectiveness  (Bennett,  1983;  Pearson  and  Shim, 
1994).  This  is  not  to  infer  that  a  DSS  that  is  more  easily 
accessible,  as  with  a  web  browser,  does  not  provide 
measurable  benefits.  Intuitively,  the  more  accessible  the 
design  interface,  the  more  positive  the  impact  will  be  on 
decision-making  effectiveness.  Because  internet  browser 
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technology  has  only  been  introduced  since  the  mid  1990' s, 
there  is  a  lack  of  research  into  the  relative  effectiveness 
of  this  interface  versus  others. 

4.  The  Role  of  Individual  Decision-Makers  and 
Organizations 

The  fourth  group  of  studies  has  addressed  the  role  of 
the  decision-maker  and  how  differences  between  individuals 
and  organizations  can  influence  the  effectiveness  of  DSS. 
Research  includes  theoretical  studies  of  organizational 
decision-making.  Specific  characteristics  investigated 
include  cognitive  biases  and  processes,  novice/expert 
effects,  models  of  decision-making,  and  user-situational 
variables  (Mittman  and  Moore,  1984;  Mann  et  al.,  1986;  Keen 
and  Morton,  1978;  Alavi  and  Joachimsthaler,  1992) .  User- 
situational  variables  include  user  involvement,  training  and 
experience.  The  emphasis  in  this  area  is  on  describing  the 
methodologies  and  differences  of  decision-making  so  that 
computer  technologies  can  be  effectively  prescribed  and 
applied  to  improve  how  decisions  are  made.  Research  has 
also  found  that  MIS  success  is  dependent  upon  the  extent  to 
which  it  fits  the  organizational  environment  (Raymond,  1990; 
Ein-Dor  and  Segev,  1978;  Ein-Dor  and  Segev,  1982;  Schultz 
and  Slevin,  1975) . 
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5.  The  DSS  Evaluation  Methods 

The  last  major  area  of  research  is  that  of  measuring 
the  implementation  success  of  the  DSS.  Implementation 
success  refers  simply  to  realizing  the  intended  benefits  of 
the  system.  Currently,  no  single  approach  to  the  definition 
of  DSS  implementation  success  exists  in  the  literature.  A 
variety  of  different  variables,  have  been  proposed  and  tested 
as  indicators  of  success.  These  include  such  things  as 
system  use,  decision-making  time,  decision-making 
performance,  user  satisfaction  with  the  system,  user 
confidence  in  the  decisions,  and  user  attitudes  towards  the 
DSS.  (Alavi  and  Joachimsthaler,  1992;  Money  et  al.,  1988; 
Raymond,  1990;  Lee  et  al.,  1995;  Swink,  1995;  Goodhue,  1995; 
Gatian,  1994) 

The  evaluation  of  DSS  is  a  research  direction  mentioned 
by  almost  every  author  in  this  field,  but  measuring  the 
effectiveness  of  these  systems  is  a  difficult  task  (Udo  and 
Davis,  1992) .  Again,  the  literature  indicates  little  or  no 
consensus  as  to  a  model  or  methodology  for  evaluating 
success.  Those  proposed  have  progressed  from  the 
traditional  cost-benefit  analysis  (King  and  Schrems,  1978) 
to  techniques  that  attempt  to  include  the  intangible  and 
qualitative  benefits  of  the  DSS. 
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First  among  these  is  Value  Analysis  (Keen,  1981;  Money 
et  al.,  1988).  An  important  premise  of  this  approach  is 
that  the  perceived  benefits  of  the  DSS  are  significant 
determinants  in  justifying  a  specific  DSS. 

Keen  and  Scott  Morton  (1978)  advocate  a  smorgasbord 
approach  to  determining  effectiveness.  Eight  methodologies 
to  be  matched  to  a  specific  situation  are  proposed.  These 
include  examining  decision  outputs;  changes  in  decision 
process;  changes  in  managers^  concepts  of  the  decision 
situation;  procedural  changes;  classical  cost/benefit 
analysis;  service  measures;  managers'  assessment  of  the 
system's  value;  and  anecdotal  evidence. 

Adelman  (1992)  suggests  that  an  eclectic  approach  is 
required  to  test  and  evaluate  DSSs  effectively.  He  defined 
three  alternative  types  of  evaluation  procedures:  objective 
measurement,  expert  observation,  and  subjective  judgment. 
Any  one  or  combination  of  these  methods  could  be  used 
depending  upon  the  system  and  what  the  system  was  to 
achieve. 

A  fundamental  aim  of  an  organizational  information 
system  (IS)  is  to  improve  individual  decision-making 
performance,  and  ultimately  organizational  effectiveness. 
The  difficult  in  empirically  assessing  system  effectiveness 
in  this  way  has  led  researchers  to  adopt  surrogate 
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constructs  that  are  more  easily  measured.  Of  the  two  main 
approaches  for  evaluating  IS  success,  the  first  one  is 
behavioral  and  focuses  on  systems  usage.  This  approach  is 
often  used  in  empirical  research  (Baroudi  et  al.,  1986; 
Gremillion,  1984;  Hogue  and  Watson,  1985;  Raymond,  1985) . 
Here  the  implication  is  that  if  the  information  system  helps 
improve  decision  quality,  then  the  end  user  will  use  the 
system. 

The  second  approach  in  evaluating  success  centers  on 
user  attitudes,  more  specifically  on  user  satisfaction  with 
various  aspects  of  an  information  system  (Lee,  et  al.,  1995; 
Gatian,  1994;  Swink,  1995;  Hogue  and  Watson,  1985).  End 
user  IS  satisfaction  is  the  extent  to  which  users  believe 
the  system  meets  their  information  requirements  (Ives,  et 
al.,  1983).  IS  satisfaction  is  assumed  to  be  a  good 
substitute  for  objective  determinants  of  information  system 
success.  The  basic  idea  is  that  satisfied  users  should 
perform  better  than  dissatisfied  users  and  if  the  IS  helps 
users  perform  better,  the  system  is  successful  (Gatian, 
1994). 

Other  research,  going  beyond  the  user  satisfaction  with 
the  system,  has  focused  on  satisfaction  with  information 
quality.  Gatian' s  (1994)  findings  support  the  theory  that 
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availability  of  relevant  information  improved  decision 
performance . 

Yet  another  method  that  has  been  proposed  looks  at  the 
process.  This  methodology  attempts  to  trace  the  effects  of 
the  system  through  all  stages  of  the  decision  process.  The 
focus  is  on  the  outcomes  of  the  process  and  its  individual 
steps  (Vetschera  and  Walterscheid,  1995) . 

We  believe  that  a  combination  of  these  methods  is 
necessary  to  determine  the  success  of  a  system.  A  model  that 
combines  the  process  orientation  with  the  information 
quality  methods  is  that  of  task-technology  fit  (Goodhue, 
1995) .  The  essence  of  this  model  is  that  task-technology 
fit  is  presumed  to  lead  to  higher  performance.  Systems  that 
provide  information  necessary  to  a  user's  tasks,  at  the 
right  level  of  detail,  clearly  and  unambiguously  will  be 
highly  valued.  We  intend  to  take  this  research  one  step 
further  and  apply  it  to  a  specific  DSS. 

B.  MEA.SURIN6  LIFE  CYCLE  COSTS 

The  LMDSS  has  been  identified  as  a  tool  to  reduce  life 
cycle  costs  (LMDSS  Req.  Doc.,  1993).  In  this  section  we 
provide  a  review  of  literature  to  support  the  significance 
of  measuring  life  cycle  cost  versus  alternative  program 
measures . 
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The  recent  combination  of  economic  trends,  rising 
inflation,  products  and  system  cost  growth,  the  continued 
reduction  of  buying  power,  and  budget  limitations  has 
increased  the  awareness  and  interest  in  total  system  cost. 
Not  only  are  the  acquisition  costs  associated  with  new 
systems  rising,  but  the  costs  of  operating  and  maintaining 
systems  already  in  use  are  increasing  at  alarming  rates 
(NAVAIR  TEAM,  1998;  Hickock,  1997).  The  requirement  to 
increase  overall  productivity  in  a  resource-constrained 
environment  has  placed  emphasis  on  all  aspects  of  the  system 
or  product  life  cycle.  In  the  past,  total  system  cost  has 
not  been  readily  visible,  particularly  those  costs 
associated  with  system  operations  and  support.  As  these 
cost  elements  are  increasingly  visible  through  computerized 
tracking  and  activity-based  costing,  they  can  more  readily 
be  managed.  Further,  when  addressing  total  cost,  experience 
has  shown  that  a  major  portion  of  the  projected  life-cycle 
cost  for  a  given  system  or  product  is  a  result  of  the 
consequences  of  decisions  made  during  the  early  phases  of 
program  planning  and  system  conceptual  design  (Blanchard, 
1992)  .  Decisions  at  this  point  have  a  major  impact  on 
activities  and  operations  in  all  subsequent  phases  of  the 
life  cycle. 
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Blanchard  (1992)  relates  the  cost  visibility  problem  to 
the  "iceberg  effect."  The  acquisition  cost  of  research, 
design,  test,  production,  and  construction  are  visible  above 
the  water.  The  mass  of  the  iceberg  below  the  surface 
illustrates  additional,  less  visible  costs  such  as: 

•  operations  cost  (personnel,  facilities,  utilities, 
and  energy) ; 

•  product  distribution  cost  (transportation, 
traffic,  material  handling) ; 

•  software  cost  (operating  and  maintaining 
software) ; 

•  maintenance  cost  (consumer  service,  supplier 
factory  maintenance) ; 

•  test  and  support  equipment  cost; 

•  technical  data  cost; 

•  supply  support  cost  (spares,  inventory,  material 
support) ; 

•  training  cost  (operator  and  maintenance  training) ; 

•  retirement  and  disposal  cost. 

The  greatest  opportunity  to  influence  total  cost,  which 
is  predominantly  made  up  of  the  costs  illustrated  as  those 
costs  below  the  water,  is  during  the  early  phases  of  a 
program.  Decisions  relating  to  the  evaluation  of 
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alternative  operational  use  profiles,  maintenance  and 
support  policies,  equipment  packaging  and  transportation 
schemes,  and  level  of  repair  concepts  have  a  great  impact  on 
total  cost.  An  overarching  goal  is  to  field  high-quality 
products,  systems,  and  structures  in  response  to  established 
needs.  It  is  through  a  concurrent  life-cycle  approach  that 
managers  can  deal  with  all  economic  factors  (Fabrycky  and 
Blanchard,  1991)  and  recognize  the  life-cycle  implication 
associated  with  almost  all  decisions.  Efficient  and 
effective  decisions  result  from  analysis  of  the  total 
program  (cost,  performance,  schedule,  and  political 
elements)  relative  to  the  total  life  cycle  (concept  design 
and  requirements  planning,  design  and  development, 
production,  utilization,  and  retirement/disposal) . 

Product  or  system  life-cycle  analysis  can  be  applied  to  the 
evaluation  of  numerous  alternatives,  including: 

1)  operational  scenarios  and  utilization  approaches; 

2)  system  maintenance  concepts  and  logistic  support 
policies; 

3)  design  configurations,  technology  applications, 
built  in  test  versus  external  test,  reliability 

'  versus  maintainability,  or  levels  of  repair  versus 
discard  decisions; 

4)  supplier  sources; 
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5)  number  of  inventory  points  and  levels  of  inventory, 
transportation  and  handling  methods; 

6)  inspection  and  test  policies;  and 

7)  product  recycling  and  disposal  methods.  (Fabrycky 
and  Blanchard,  1991) 

We  follow  Fabrycky  and  Blanchard  (1991)  in  holding  that 
an  important  first  step  in  the  analysis  is  clarifying  the 
analysis  objectives.  It  is  important  to  define  the  issues 
of  concern  and  bind  the  problem  such  that  the  study  can  be 
efficient.  Too  large  an  effort  can  become  overwhelming  and 
it  is  easy  to  proceed  in  the  wrong  direction.  The  problem 
must  be  defined  clearly,  precisely,  and  presented  in  such  a 
manner  as  to  be  easily  understood  by  all  concerned. 
Otherwise,  it  is  unlikely  an  analysis  of  any  kind  will  be 
meaningful.  Within  the  established  bounds  and  constraints, 
all  possible  alternatives  should  be  considered,  with  the 
most  likely  candidates  selected  for  further  evaluation. 
Alternatives  not  considered  cannot  be  adopted;  therefore,  it 
is  better  to  consider  all  candidates  even  those  that  do  not 
seem  attainable  or  likely  rather  than  overlook  one  that  may 
be  good. 

One  of  the  greatest  challenges  facing  industry, 
government  agencies,  the  Department  of  Defense,  and  the 
general  consumer  of  products  and  services  is  the  growing 
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need  for  more  effective  and  efficient  management  of  our 
resources.  The  Department  of  Defense  logistics  mission  is 
"to  provide  responsive  and  cost-effective  support  to  ensure 
readiness  and  sustainability  for  the  total  force  in  both 
peace  and  war."  (USD(A5eT),  1998)  The  fact  that  logistics 
costs  incurred  during  the  operating  and  support  phase  are 
such  a  large  part  of  total  cost  requires  logistics  to  assume 
a  major  role  during  operational  use.  Given  the  cause-and- 
effect  relationships  between  early  planning  and  later  costs, 
logistics  has  become  equally  significant  in  every  phase  of 
the  life  cycle.  For  these  reasons  research,  design, 
production,  logistics,  and  system  performance  analyses  must 
be  addressed  early,  concurrently  performed,  and  integrated 
throughout  the  system  or  product  life  cycle. 

The  above  discussion  demonstrates  the  value  of  measuring 
life  cycle  costs  and  the  value  of  logistics  in  life  cycle 
cost  analysis.  The  challenge  then  becomes  how  to  measure 
and  evaluate  logistics  performance.  Good  measurements  should 
cover  all  aspects  of  the  process  being  measured,  be 
appropriate  for  each  situation,  minimize  measurement  error, 
and  be  consistent  with  the  management  reward  system  (Menzer 
and  Konrad,  1991) .  Fabrycky  and  Blanchard  (1991)  recommend 
the  cost  breakdown  structure  (CBS)  to  provide  a  framework 
for  defining  life— cycle  costs  and  communicating  links  for 
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cost  reporting,  analysis  and  cost  control.  CBS  is  a  way  of 
classifying  costs  with  the  classification  being  life-cycle 
oriented.  The  CBS  links  objectives  and  activities  with 
resources,  and  sets  up  a  logical  subdivision  of  cost  by 
functional  activity  area  and  major  element  of  a  system.  It 
can  be  used  as  a  basis  for  assessing  the  life-cycle  cost  of 
each  alternative  being  considered.  In  logistics  management, 
as  with  other  management  decisions,  optimal  solutions  are 
often  based  on  more  than  simply  the  financial  bottom  line. 
First  and  foremost  systems  must  measure  up  to  operational 
demands.  The  decisions  that  support  those  demands  must  also 
be  fiscally  responsible. 
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V.  DATA  COLLECTION  TECHNIQUES  AND 

METHODOLOGY 


A.  CONDUCT  OF  THE  STUDY 

Several  techniques  are  commonly  used  to  obtain 
perceptions,  opinions,  and  judgments  from  subject  matter 
experts.  These  generally  fall  into  two  categories:  personal 
interviews  and  questionnaires.  (Adelman,  1992)  Both  of  these 
techniques  were  employed  in  the  course  of  this  study. 
Telephone  interviews  were  conducted  with  logistics  managers. 
These  interviews  were  directed  and  structured,  but  allowed 
for  open-ended  responses  in  a  number  of  specific  areas. 

A  structured  questionnaire  and  copy  of  the 
interviewer's  notes  from  the  telephone  interview  were  sent 
to  each  respondent  after  the  telephone  interview.  Included 
were  self  addressed  and  stamped  envelopes  for  the  surveys  to 
be  mailed  back.  We  also  used  follow-up  e-mail  and  telephone 
calls  in  an  effort  to  improve  the  response  rate. 

We  were  also  fortunate  enough  to  be  able  to  spend  a 
week  at  NAVAIR.  During  that  time,  we  were  able  to  conduct 
face-to-face  interviews  with  the  PMA  representatives  that  we 
had  been  unable  to  reach  by  telephone.  Additionally,  we 
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were  afforded  extensive  briefings  on  the  LMDSS  capabilities, 
structure,  data  elements,  development  and  history. 

B .  THE  SAMPLE 

We  selected  logistics  managers  as  targets  for  our 
study.  This  group  has  been  specifically  identified  by  the 
LMDSS  development  team  as  the  targeted  users  of  the  LMDSS. 
There  are  a  total  of  twelve  aircraft  types,  support 
equipment,  and  engine  program  management  teams  considered 
relevant  to  the  study. 

The  primary  target  within  each  PMA  was  the  Assistant 
Program  Manager  for  Logistics  (APML) .  In  some  cases,  we 
were  referred  to  the  deputy  APML,  Product  Support  Team  Leads 
or  other  support  logisticians  due  to  schools,  retirement, 
travel  or  simply  to  provide  yet  another  perspective. 

C .  QUESTIONNAIRE  DEVELOPMENT 

Our  goal  was  to  develop  a  questionnaire  that 
ascertained  the  task-technology  fit.  In  order  to  develop  a 
questionnaire  that  assessed  the  appropriate  areas  a  pre¬ 
study  was  conducted.  We  reviewed  checklists  and 
instructions  used  by  logisticians,  interviewed  three 
logisticians  and  examined  the  LMDSS  data  elements. 
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The  nature  of  this  DSS  evaluation  was  specific. 

Because  we  wished  to  elicit  responses  on  the  usefulness  of 
system  characteristics  specific  to  the  LMDSS,  not  an 
abstract  system,  an  off-the-shelf  survey  instrument  was  not 
considered  appropriate  to  our  needs. 

A  four  part  instrument  was  prepared.  The  first  part 
was  to  be  conducted  as  an  interview  with  the  logistics 
manager.  This  section  was  designed  to  elicit  information 
about  the  tasks,  decision  environment /process,  and 
information  needs  of  the  logistician. 

The  next  three  parts  were  designed  to  be  filled  out  by 
the  respondent.  Part  Two  was  to  be  filled  out  by  current 
users  of  the  LMDSS.  It  called  for  responses  in  Likert-type- 
scales.  See  Appendix  F.  Questions  were  separated  as  to 
general  logistics  concerns  and  specific  LMDSS  queries. 
Additionally,  user  evaluation  of  specific  LMDSS  functions 
and  data  was  requested. 

Part  Three  was  designed  for  non-users  of  LMDSS.  This 
section  was  identical  to  the  general  logistics  concern 
section  of  Part  Two.  In  addition,  a  checklist  of  possible 
reasons  for  not  using  the  LMDSS  was  presented  with  direction 
to  check  all  that  apply. 

Part  Four  was  applicable  to  both  users  and  non-users  of 
the  LMDSS.  This  section  also  employed  Likert-type  scales  to 
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elicit  responses  on  job  information  needs.  Listed  were  the 
information  elements  supported  by  the  LMDSS.  The  respondent 
was  asked  to  rate  the  usability  of  each  element. 

Since  this  instrument  had  not  been  validated  by 
previous  research,  we  pretesting  the  survey  with  experienced 
logisticians.  Based  on  their  remarks,  we  made  minor 
improvements  before  conducting  the  phone  survey  and  then 
mailing  the  questionnaire  to  participants.  The 
questionnaire  is  available  in  Appendix  F. 

D.  DATA  ANALYSIS  STRATEGY 

The  survey  results  consisted  of  frequency,  capacity, 
satisfaction  or  usability  ratings  assigned  to  166  individual 
factors  by  survey  participants.  Data  were  compiled  from  the 
eight  survey  forms  and  entered  into  the  Microsoft  Excel 
spreadsheet  program. 

For  each  question  a  frequency  of  total  respondents 
selecting  a  particular  rating  was  recorded.  Trends  in  the 
data  (that  is,  rank  orderings)  were  determined  instead  of 
making  direct  comparisons  of  adjacent  ratings,  due  to  the 
small  sample  available  for  the  survey. 
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VI .  FINDINGS 


A.  APML  QUESTIONNAIRES 

Table  VI-1  lists  the  APML  point  of  contact  by  title  and 
the  method  by  which  data  was  received.  Of  the  twelve  total 
aircraft,  support  equipment,  and  engine  program  management 
teams  considered  relevant  to  the  study  we  were  successful  in 
communicating  with  nine.  The  results  of  the  written 
questionnaire  are  provided  in  Appendix  G.  Seven  of  the 
eight  written  questionnaires  received  were  from  non-users  of 
the  LMDSS.  We  interviewed  the  E-2C  APML  and  EA-6B  APMLs  but 
did  not  receive  a  completed  written  questionnaire.  They 
both  do  not  use  the  LMDSS  directly,  but  receive  reports  from 
data  analysts  who  use  the  LMDSS  and  other  tools.  All 
respondents  are  aware  of  the  functions  of  the  LMDSS. 

The  APMLs  were  selected  as  targets  for  our  study.  This 
group  has  been  specifically  identified  as  the  targeted 
users  of  the  LMDSS.  We  were  referred  to  additional  analysts 
and  logistics  specialists  by  eight  of  the  nine  APML 
representatives  interviewed  as  a  result  of  non-standardized 
organization  of  the  PMA  offices  and  non-standardized 
logistics  input.  Six  of  nine  logistics  managers  interviewed 
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employ  civilian  contractors  to  provide  reports  and  analyses. 
Two  of  nine  use  NAVAIR  logistics  specialists. 


PMA 

Aircraft  System 

Point  of 

Method  of  Contact 

Contact 

PMA-222 

Electronic  Warfare  and 
Simulation,  Aircraft  Engine 
and  Special  Mission  Aircraft 

NA 

Not  contacted  due  to 
retirement  of  APML 

PMA-225 

H-3,  T-2,  A-4 

Logistics 
Management 
Specialist 
T-2 /A-4 

phone  interview  and 
written  questionnaire 

PMA-226 

H-46,  C-130,  F-4 

NA 

Not  contacted  -  could 
not  locate  good  phone 
number 

PMA-231 

E-2,  C-2 

APML  E-2C 

Personal  interview 

PMA-234 

A-6/EA-6  Intruder/Prowler 

APML  EA-6B 

phone  interview  and 
partial  written 
questionnaire 

PMA-241 

F-14  Tomcat 

Deputy  APML 

phone  interview  and 
written  questionnaire 

PMA-260 

Aviation  Support  Equipment 

PMA 

phone  interview  and 
written  questionnaire 

PMA-261 

C/MH-53E  and  Executive 
Transport  Helicopter 

APML  H-53 

phone  interview  and 
written  questionnaire 

PMA-265 

A  F/A-18  Hornet 

APML 

phone  interview  and 
written  questionnaire 

PMA-276 

Light /Attack  Helicopter 

Deputy  APML 
for  HI 
upgrades 

phone  interview  and 
written  questionnaire 

PMA-290 

Maritime  Surveillance 

Aircraft  (MSA) 

Logistics 

Management 

Specialist, 

P-3 

1  Personal  and  phone 
'  interview;  written 
questionnaire 

PMA-299 

H-2/H’-60  Multi-Mission 
Helicopter 

NA 

APML  away  at  school. 

Tedsle  VI -1  FMA  Points  of  Contact 


Six  of  nine  respondents  work  with  both  new  aviation 
systems  and  sustaining  existing  systems.  One  works  only 
with  new  aviation  systems  and  two  work  only  with  sustaining 
existing  systems.  Table  VI-2  shows  the  frequency  and 
capacity  respondents  work  with  a  number  of  tools  while 
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performing  their  job.  Table  VI-3  shows  the  frequency  and 
capacity  with  which  respondents  interface  with  project 
engineers,  analysts,  those  who  influence  the  budget,  and 
others  while  performing  their  job. 

The  responses  of  the  single  respondent  who  uses  the 
LMDSS  is  provided  in  Table  VI-4,  Table  VI-5,  and  Table  VI-6 
that  summarize  the  user-unique  questions.  The  responses  of 
the  non-user  respondents  to  non-user  unique  questions  are 
summarized  in  Table  VI-7. 

All  responses  are  included  in  the  data  in  Table  VI-8 
Experience  With  Data  Sources  Other  Than  LMDSS,  Table  VI-9 
Logistics  Concerns,  and  Table  VI-10  Job  Information  Needs. 

B.  RDBMS  OR  DSS? 

In  Chapter  IV  (Lit  Review)  we  presented  a  detailed  DSS 
definition.  A  comparison  of  the  LMDSS  with  this  definition 
leads  to  the  following  findings: 

1.  Data  Management  Con^onent 

The  LMDSS  fully  meets  these  component  criteria.  The 
NALDA  II  Oracle  RDBMS  creates  an  IDE  with  multiple  data 
sources.  Ad  hoc  query  capability  -  while  limited  to  certain 
users  -  is  available. 

2 .  Model  Cos^onent 

The  LMDSS  has  no  modeling  capability.  The  LMDSS  does 
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TOOL 

Logistics  Support  Analysis 
(MIL-STD-1388) 

Raw  Data 
Models 
Checklists 
Intuition/Experience 


Table  VI-2  Logistics  Management  Tools 


INTERFACE  FREQUENCY 

CAPACITY 

JS 

s 

i 

CO 

£ 

i 

1 

# 

1 

(D 

C 

# 

1 

e 

is 

i 

O 

# 

1 

0 

M 

<0 

i 

# 

Interface  with  project  engineers 

0 

0 

0 

0 

1 

0 

0 

1 

7 

0 

Interface  with  project  analysts 
Interface  with  those  who 

0 

0 

0 

0 

1 

0 

0 

1 

6 

0 

influence  the  budget 

0 

1 

0 

0 

0 

0 

0 

0 

4 

3 

Interface  with  others: 

Depot  Maintenance 

0 

0 

0 

1 

3 

0 

0 

0 

4 

0 

Publication  Coordinators 

Type  Commanders/ 

0 

0 

0 

1 

1 

0 

0 

0 

2 

0 

Functional  Wings 

0 

0 

0 

1 

2 

0 

0 

0 

3 

0 

NAVICP 

0 

0 

0 

1 

3 

0 

0 

0 

4 

0 

Suppliers  (commercial,  organic) 
integrated  Logistics 

0 

0 

0 

0 

3 

0 

0 

0 

3 

0 

Support  specialists 

0 

0 

0 

1 

3 

0 

0 

0 

4 

0 

Contracts  Office 

0 

0 

0 

1 

0 

0 

1 

0 

0 

0 

FREQUENCY  OF  USE  CAPACITY  OF  USE 


3  4  1  0  0  0  1  1  2  1 

0  0  0  1  7  0  0  1  6  1 

400021  0  031 

3  00040  0  0  40 

000080  0  1  70 


Table  VI-3  Logistics  Management  Interfaces 
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THE  LMDSS  QUERY 

FREQUENCY  OF  USE 

Summary  data;  End  item 

daily 

Reliability  summary  parameters 

daily 

Supportability  summary  parameters 

daily 

Cost  summary  parameters 

dally 

Trend  analysis  (problems  and  causes) 

daily 

Component  and/or  end  item  cost  data; 

specifically: 

Annual  Operations  and  Support  Costs 

daily 

Labor  Cost  History 

daily 

Item  Value  to  Depot  Repair  Cost 

daily 

Budget  Analysis  (OP-20  Report) 

infrequently 

Inflation  Factors 

infrequently 

Item  Value  to  Labor  Cost 

infrequently 

Candidate  Identification  Function 

specifically: 

Detailed  Component  Report 

daily 

Wholesale  System  Demand 

infrequently 

Material  Issue  Trends 

daily 

Supply  Synopsis 

infrequently 

Wholesale  System  Investment 

infrequently 

Average  Customer  Wait  Reports 

infrequently 

Backorder  History  Reports 

infrequently 

NAVICP  NSN  Snapshot 

daily 

Mean  Flight  Hour  Between  Failure  Report 

daily 

Engine  Repair  Cost 

Infrequently 

Flight  Hours  Since  Last  Engine  Repair 

infrequently 

Engine  Demand  Forecasting 

infrequently 

Engine  Overview 

infrequently 

Reference  Information,  specifically: 

Aircraft  Inventory  and  Fatigue  Life 

infrequently 

Code  Definition 

infrequently 

Aircraft  Engine  Installation  and  Ownership 

infrequently 

Production  Load  and  Run  Statistics 

infrequently 

Possible  Courses  of  Action 

infrequently 

Organization  Codes/Job  Count 

daily 

NIIN/CAGE/Part  Number  Cross  Reference 

daily 

TEC  Information 

daily 

SALTS  File  informatiori 

infrequently 

Data  Dictionary 

Infrequently 

Table  VI -4  The  I14DSS  Queries 


FUNCTION 


RESPONSE 


Interface 

like 

Help  Function 

like 

Analysis  Tools 

like 

Report  Presentation/Format 

like 

Time  Required  to  get  what  is  needed 

dislike 

Ease  of  getting  what  is  needed 

dislike 

Training 

strongly  dislike 

Accessibility  (when  desired) 

neutral 

Accessibiiity  (server  access) 

neutral 

Accessibility  (password  access) 

like 

Provides  information  needed 

like 

Table  VI-5  The  LMDSS  Functions 


EXPERIENCE 

RESPONSE 

Data  meet  needs 

agree 

Data  accessible 

agree 

Data  accurate/consistent 

strongly  disagree 

Data  detailed  enough 

agree 

The  exact  data  meaning  is  clear 

disagree 

Table  VI-6  Esqperience  with  the  UIDSS  Data 
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NUMBER  OF  RESPONSES 


Didn’t  know  it  existed  2 

PC  won’t  support  the  LMDSS  1 

Received  no  training  4 

Don’t  need  the  information  the  LMDSS  provides  0 

It  takes  too  long  to  get  what  is  needed  0 

It’s  too  difficult  to  get  what  is  needed  0 

It  doesn’t  provide  the  information  needed  2 

Other  Not  developed  for  logistics  (everyday)  Issues  yet  1 


Table  VI-7  Why  the  LMDSS  is  not  used 


EXPERIENCE 


RESPONSE 


Data  meet  needs 

Data  accessible 

Data  accurate/consistent 

Data  detailed  enough 

The  exact  data  meaning  is  clear 


0 

0 

0 

0 

0 


4  0  2  8  0 

1  3  2  6  2 

1  3  4  5  1 

2  3  5  3  1 

4  2  3  3  2 


Table  VI-8  Experience  with  Data  other  than  the  IMDSS 
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FREQUENCY  OF  USE 


CAPACITY  OF  USE 


LOGISTICS  CONCERN 


I  / 


‘c 

I  5  « 


Logistics  criteria  as  Input  to  systems  design 
Human  factors  concerns 
Failure  mode,  effects,  and  critical  analysis 
Failure  reporting,  analysis,  and 
corrective-action  system 
Provisioning  needs/altematives 
Compatibility  with  existing  system 
Configuration  Management 
Training  and  training  support 
Manpower  and  personnel 
Supply  Support,  spares 
Inventory  level  analysis 
Transportation,  packaging  or  storage 
Test  equipment 
Support  equipment 
Computer  resource  support 
Facilities,  requirements 
Facilities,  location 
Data,  reports  requirements 
Maintenance  planning 

(scheduled  versus  unscheduled) 

Level  of  repair  analysis 
(O  versus  I  versus  D  level) 

Operating  environment  issues 

Cost-drivers 

Readiness  degraders 

Cycle  time  to  repair  components 


0  3  2 

0  5  1 

0  5  0 


0  3  0 
0  2  0 
1  2  0 
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it/op 


HOW  USEFUL  IS  THE  INFORMATION  ELEMENT?  I 

JOB  INFORMATION  ELEMENT 

^ 

.A- 

y 

Summary  data;  End  item 

0 

0 

0 

2 

4 

1 

Summary  data;  Claimant 

0 

0 

2 

1 

2 

2 

Summary  data;  Organization 

0 

1 

0 

1 

5 

0 

Summary  data;  BCM  Report 

0 

1 

0 

2 

4 

0 

Reliability  summary  parameters 

0 

0 

0 

1 

6 

0 

Supportability  summary  parameters 

0 

0 

0 

1 

5 

1 

Cost  summary  parameters 

0 

0 

0 

1 

6 

0 

Emerging  problems 

0 

0 

0 

0 

7 

0 

Common  Equipment 

0 

0 

0 

1 

4 

2 

Trend  analysis  (problems  and  causes) 

0 

0 

0 

1 

5 

1 

Component  and/or  end  item  cost  data;  specifically: 

Annual  Operations  and  Support  Costs 

0 

0 

0 

0 

7 

0 

Labor  Cost  History 

0 

0 

0 

0 

7 

0 

Item  Value  to  Depot  Repair  Cost 

0 

0 

0 

0 

5 

2 

Budget  Analysis  (OP-20  Report) 

0 

0 

1 

2 

3 

1 

Inflation  Factors 

0 

0 

1 

3 

2 

1 

Item  Value  to  Labor  Cost 

0 

0 

0 

2 

3 

2 

Candidate  Identification  Function;  specifically: 
Detailed  Component  Report 

0 

0 

2 

0 

4 

1 

Wholesale  Sykem  Demand 

0 

0 

1 

4 

1 

1 

Material  Issue  Trends 

0 

0 

1 

2 

3 

1 

Supply  Synopsis 

0 

0 

1 

1 

4 

1 

Wholesale  System  Investment 

0 

0 

1 

3 

2 

1 

Average  Customer  Wait  Reports 

0 

0 

1 

3 

3 

0 

Backorder  History  Reports 

0 

1 

1 

2 

3 

0 

NAVICP  NSN  Snapshot 

0 

1 

0 

2 

4 

0 

Mean  Flight  Hour  Between  Failure  Report 

0 

0 

0 

1 

5 

0 

Wait  Time  Maintenance  Impact 

0 

0 

1 

2 

3 

1 

Average  Days  to  Receipt 

0 

0 

1 

3 

3 

0 

Planned  versus  Actual  Opportunity  Costs 

0 

0 

1 

2 

2 

2 

Engine  Repair  Cost 

0 

0 

0 

0 

5 

1 

Flight  Hours  Since  Last  Engine  Repair 

0 

0 

0 

1 

4 

1 

Engine  Demand  Forecasting 

0 

0 

1 

0 

4 

1 

Engine  Overview 

0 

0 

1 

1 

2 

2 

Engine  Removal  Trend 

0 

0 

1 

0 

4 

1 

Flight  Hours  Since  Engine  Repair  at  Removal 

0 

0 

1 

0 

3 

2 

Reference  Information,  specifically: 

Aircraft  Inventory  and  Fatigue  Life 

0 

1 

1 

1 

3 

0 

Code  Definition 

0 

0 

1 

2 

2 

1 

Aircraft  Engine  Installation  and  Ownership 

0 

1 

1 

1 

2 

1 

Production  Load  and  Run  Statistics 

0 

0 

1 

4 

1 

1 

Possible  Courses  of  Action 

0 

0 

1 

0 

4 

2 

Organization  Codes/Job  Count 

0 

1 

2 

2 

2 

0 

NIIN/CAGE/Part  Number  Cross  Reference 

0 

0 

1 

2 

4 

0 

TEC  Information 

0 

0 

1 

3 

3 

0 

SALTS  File  Information 

1 

0 

3 

1 

0 

2 

Data  Dictionary 

0 

0 

0 

4 

1 

2 

Table  VI-10  Logis'txcs  Management:  Information  Elements 
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offer  a  module  entitled  Trend  Analysis.  This  module 
provides  historical  data  in  tabular  format.  Historical  data 
is  also  presented  in  a  format  that  could  support  time-series 
forecasting.  This  is  found  in  Engine  Demand  Forecasting  in 
the  Engine  Analysis  module,  and  Wholesale  System  Demand  in 
the  Supply  Analysis  module.  Also  within  the  Supply  Analysis 
module  are  two  subareas  that  can  accept  manual  entries  in 
some  parameters.  This  allows  for  a  degree  of  "what  if" 
analysis  for  Mean  Flight  Hour  Between  Failures  Report  and 
Planned  vs  Actual  Opportunity  Cost.  However,  there  is  no 
sensitivity  analysis  capability  available  to  support  the 
"what  if"  analysis. 

3.  Dialogue  Management  Component 

The  LMDSS  meets,  to  some  degree,  all  of  the  criteria  of 
this  component.  The  browser  interface  and  hyperlinks  offer 
navigational  flexibility.  Usage  of  the  OLAP  tool  provides 
an  additional  degree  of  flexibility.  Output  either  can  be 
to  the  screen  in  HTML  format  or  can  be  e-mailed  to  the 
requester  in  a  format  that  can  be  imported  to  a  spreadsheet 
application.  There  is  no  graphics  capability. 

C .  DATA  QUALITY 

In  the  course  of  this  study,  three  data  quality  issues 
were  identified. 
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1 .  Data  Accessibility 

The  IDE  improves  accessibility  of  data.  The 
LMDSS/NALDA  II  database  reduces  the  necessity  of  querying 
multiple  disparate  databases  to  retrieve  relevant  data  for 
analysis.  An  economic  analysis  undertaken  as  part  of  the 
NALDA  II  Milestone  III  approval  process  found  that  the  new 
system  will  allow  all  potential  users  to  substantially 
reduce  the  amount  of  time  required  to  identify  and  analyze 
problems  in  logistics  support  by  incorporating  data  from  as 
many  as  nine  other  disparate  databases  which  are  currently 
used  for  analysis.  A  cost  avoidance  of  $2  million  is 
expected  over  the  life  cycle  of  the  LMDSS.  In  the  area  of 
Common  Equipment  Analysis,  there  is  a  potential  cost 
avoidance  of  $19  million.  The  IDE  allows  for  performance 
and  reliability  of  specific  components  across  the  whole 
spectrum  of  Naval  aircraft  to  be  ascertained.  This  data 
accessibility  did  not  exist  prior  to  the  LMDSS.  (NAVAIR  7.0, 
1997) 

There  is,  however,  misunderstanding  on  the  part  of  many 
logistics  representatives  of  this  data  accessibility.  A 
perception  exists  that  the  LMDSS  and  NALDA  II  will  hamper  or 
eliminate  the  analyst's  ability  to  access  detail  data  (NAWC 
AD  3. OB,  1998).  The  Support  Equipment  PMA  believed  that  the 
Cost  Analysis  capability  would  be  inadequate  for  their  needs 
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because  cost  data  from  DLA  was  not  included.  A  review  of 
the  external  interface  data  for  NALDA  II  indicates  that 
extensive  DLA  cost  data  will  be  available  (Capstone,  1997). 

2 .  Data  Consistency 

Data  consistency  has  two  aspects.  Consistency  between 
data  retrieved  under  NALDA  I  and  NALDA  II,  and  consistency 
between  modules  in  the  LMDSS. 

Although  the  database  which  the  LMDSS  now  reaches  - 
NALDA  II  -  is  based  on  a  data  pull  from  NALDA  I,  the  data 
outputs  are  not  identical.  This  is  because  in  the  LMDSS, 
the  data  pre-processing  has  been  improved  to  provide  greater 
accuracy.  Examples  of  the  differences  include  use  of 
aircraft  versions  in  addition  to  TMS  and  revised  item  count 
logic. 

Currently,  there  are  identified  inconsistencies  between 
the  LMDSS  modules.  These  are  being  addressed  through  the  on 
going  quality  assurance  process  (Jones,  1998;  SCR  Sub- 
module,  LMDSS  application) . 

3.  Data  Validity 

Data  validity  was  a  concern  expressed  by  75%  of  survey 
respondents  and  interviewees.  The  general  perception  was 
that  data  validity  was  currently  poor  and  would  continue  to 
be  poor. 
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One  example  that  was  offered  by  the  SE  PMA  to 
illustrate  this  issue  related  to  Maintenance  Level  1 
(organizational  level) ,  Maintenance  Action  Forms  (MAFs)  for 
tow  tractors. 

•  47%  of  MAFs  recording  down  time  due  to  Awaiting 
parts  had  no  failed  parts  documented 

•  32%  of  MAFs  attributed  the  failed  part  to  the  part 
number  of  the  tow  tractor 

•  272  MAFs  recorded  removal  and  replacement  of  the  tow 
tractor  part  number. 

Problems  like  this  arise,  in  part,  because  finding  the 
correct  work  unit  code  (WUC)  or  part  number  can  be  a  time 
consuming  task.  Busy  maintainers  memorize  a  few  key  WUCs 
and  part  numbers  and  use  those  regardless  of  the  real 
discrepancy.  Lack  of  training  on  the  importance  of  data 
validity  may  also  contribute. 

Another  example  was  offered  by  the  logistics  management 
specialist  for  the  S-3  aircraft.  A  data  query  to  identify 
readiness  degraders  resulted  in  identifying  the- airframe  as 
the  top  system  degrader.  Further  investigation  showed  this 
was  a  result  of  how  scheduled  maintenance  washes  were  being 
documented  in  an  aircraft  squadron.  Additional  stories  were 
prevalent  of  the  adverse  impact  of  the  use  of  inaccurate 
type  equipment  codes  (TEC)  or  WUC  on  MAFs  by  maintenance 
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technicians  who  do  not  understand  how  the  data  is  used. 
Seasoned  data  analysts  know  where  to  look  to  find  erroneous 
data  and  know  how  to  remove  misleading  data  from  logistics 
management  reports.  These  data  validity  problems  are  caused 
by  poor  documentation  at  the  source.  This  problem  is 
currently  being  addressed  by  improved  Validation 
Specifications  at  the  input  point  of  NALCOMIS. 

A  second  data  validity  area  is  cost  data.  The 
maintenance  cost  data  is  incomplete.  Maintenance  Level  3 
(depot)  was  described  during  one  interview  as  a  "black 
hole."  The  only  aircraft  cost  data  for  MLS  in  the  database 
is  aggregate  cost.  It  is  not  broken  out  by  TMS.  Engine 
overhaul  cost  data  —  what  MLS  charges  the  fleet  —  is 
available  by  TMS.  The  LMDSS  database  has  placeholders  for 
detailed  cost  data,  but  that  data  is  currently  unavailable 
from  the  depots.  This  precludes  total  cost  visibility. 

A  final  area  of  concern  with  data  validity  is  the  new 
SALTS  processing  procedures.  When  NSLC  had  cognizance  of  the 
SALTS  data,  a  "scrubbing"  process  was  used.  Five  areas  of 
the  detail  data  received  from  the  fleet  activities  were 
compared  with  a  79  Record.  If  there  were  a  10%  or  greater 
discrepancy  all  detail  data  from  that  activity  would  be 
rejected. 
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Under  the  new  system,  the  data  will  not  be  scrubbed 
prior  to  being  loaded  into  the  database.  The  rational  for 
this  change  is  two  fold.  First  is  the  belief  that  there  is 
more  value  to  the  analysts  in  having  all  of  the  data  even  if 
a  small  percentage  of  them  are  in  error.  Under  the  old 
processing  system,  when  data  were  rejected  the  fleet 
activity  had  to  correct  the  data  and  then  resubmit.  It 
could  take  up  to  three  months  before  the  detail  data  was 
integrated  into  the  database.  Second  is  that  the  Most 
Probable  Logic  feature  used  when  summarizing  the  detail  data 
will  identify  and  correct  these  errors. 


VII.  DISCUSSION 


A.  APMLS  AS  USERS 

We  selected  the  APMLs  for  our  study  because  this  group 
has  been  specifically  identified  as  the  targeted  users  of 
the  LMDSS.  Of  the  twelve  total  aircraft,  support  equipment, 
and  engine  program  management  teams  considered  relevant  to 
the  study  we  received  seven  complete  responses  and  two 
partial  responses.  The  partial  responses  were  a  phone  and  a 
personal  interview  without  completion  of  the  written 
questionnaire.  Additionally,  we  conducted  personal  and 
phone  interviews  with  logistics  advisors  and  the  LMDSS 
program  representatives. 

Our  data  collection  was  limited  by  two  constraints. 

The  LMDSS  is  a  prototype  system  and  is  not  yet  widely  used, 
and  the  APML  is  only  one  of  many  potential  users  we 
subsequently  identified  during  the  course  of  our  research. 
Because  the  tasks  and  needs  of  different  users  vary  greatly, 
the  information  derived  from  the  study  of  APMLs  cannot  be 
used  to  interpret  the  needs  of  other  population  groups 
without  considerable  risk.  We  identified  the  following  user 
groups  as  equally  important  as  AMPLs  to  logistics  input  to 
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aviation  program  management  decisions:  IPT  data  analysts, 

FST  data  analysts,  and  Type  Commands  data  analysts.  These 
analysts  may  be  Navy  data  analysts,  government  employees,  or 
civilian  contractors  assigned  within  the  teams. 

As  previously  discussed,  program  management  and  the 
logistics  input  thereto  are  not  standardized.  Each  PMA 
determines  how  he  will  manage  his  program.  The  organization 
may  primarily  be  within  the  program  office,  the  aircraft 
controlling  custodian,  or  the  depot  maintenance  engineering 
support  organizations.  Additionally,  program  offices  use  a 
number  of  different  sources  for  logistics  support.  Some 
program  managers  rely  heavily  on  Navy  personnel  assigned 
within  the  program  office,  others  rely  on  government 
employees  from  a  logistics  competency  group  in  NAVAIR,  while 
some  contract  out  to  commercial  sources  for  logistics 
analyses  and  recommendations.  Furthermore,  PMAs  must 
interface  with  support  environments  such  as  policy,  process, 
facility  and  infrastructure  organizations  to  develop  optimal 
policies  and  processes;  report  actions  taken  and  results; 
and  obtain  fleet  feedback  on  system  performance.  All  of 
these  activities  point  to  additional  users  of  naval  aviation 
logistics  data  and  the  LMDSS. 

When  describing  the  APML  job,  respondents  commonly 
referred  to  designing,  developing,  or  analyzing  support 
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infrastructures  for  aircraft  and  aircraft  systems.  LTCOL 
Wiechowski,  H-53  APML,  referred  to  this  job  task  as  the 
"care  and  feeding"  of  the  heavy  lift  helos.  The  support 
infrastructure  is  not  limited  to  logistics  concerns.  As 
presented  in  Table  VI-3,  logistics  managers  interface 
frequently  (daily  or  weekly)  with  project  engineers,  project 
analysts,  those  that  influence  the  program  budget,  and 
others  to  both  identify  and  analyze  requirements. 

B.  THE  LMDSS  USE 

As  presented  in  Table  VI-7,  the  following  reasons  were 
most  frequently  cited  for  why  the  LMDSS  is  not  used. 

•  Received  no  training  -  40% 

•  Didn't  know  it  existed  -  20% 

•  It  doesn't  provide  the  information  needed  -  20% 
Training  is  a  critical  issue.  Many  studies  of  information 
technology  deployment  in  organizations  have  found  that 
failure  of  these  systems  can  be  attributed  to  the  lack  of 
relevant  and  satisfactory  training  programs  provided  for  all 
levels  of  end  users.  (Lee,  et  al,  1995;  Nelson  and  Cheney, 
1991;  Udo  and  Davis,  1992)  Training  is  an  on-going  effort  by 
the  NAVAIR  training  team.  In  addition  to  conducting 
training  for  data  analysts  located  at  Patuxent  River,  site 
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visits  to  users  located  at  other  bases  have  been  and  are 
being  conducted. 

NAVAIR  3.6.2  continues  to  advertise  the  coming  of  the 
LMDSS.  This  is  accomplished  through  the  web-site  and 
frequent  presentations  on  the  state  of  the  application.  As 
the  training  team  continues  to  reach  more  potential  users, 
awareness  of  its  existence  will  increase. 

As  pointed  out  in  the  Findings  Chapter  there  is  a  basic 
discontinuity  between  the  survey  respondents'  perception  of 
the  data  available  under  LMDSS/NALDA  II  and  what  will 
actually  be  available.  When  the  LMDSS  and  NALDA  II  are 
fully  on-line,  all  of  the  data  available  with  NALDA  I  will 
be  accessible. 

The  way  that  the  LMDSS  has  been  advertised  on  its  Web 
page  since  becoming  available  via  the  Web  has  contributed  to 
this  confusion.  The' LMDSS  Web  page  does  not  identify  the 
application  as  being  a  prototype.  It  does  not  identify  the 
database  as  being  not  fully  loaded.  This  has  led  some  _ 
respondents  to  believe  that  what  they  currently  see  is  what 
they  will  ultimately  get.  In  actuality,  the  stated 
information  needs  from  the  questionnaires  are  provided  by 
the  LMDSS  as  discussed  in  part  D  of  this  chapter. 

It  is  important  to  overcome  this  perception.  End  users 
need  to  regard  the  information  systems  they  are  using  and 
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the  information  provided  by  the  IS  as  relevant  and  useful 
for  their  job  performance,  if  they  are  to  accept  such 
systems.  (Lee,  et  al.,  1995;  Gatian,  1994) 

In  the  domain  of  DSS,  the  fundamental  role  of  computer 
support  is  to  assist  people  in  reaching  decisions  about  the 
course  of  action  to  implement  in  a  particular  problem 
situation.  A  user  must  reach  a  cognitive  state  where  he 
understands  the  issues  sufficiently  well  to  choose  to  act  or 
not.  The  computer  tools  must  be  designed  to  provide  this 
fundamental  layer  of  support  to  the  user.  The  user  has  many 
things  at  stake  -  position,  reputation,  and  self-image  -  and. 
as  such  will  rarely  be  willing  to  treat  the  computer  as  a 
black  box,  which  tells  him  what  course  of  action  to 
implement.  (Fedorowicz  and  Manheim,  1986)  With  this  in  mind, 
the  issues  of  data  validity  and  data  summarization/origin 
must  be  addressed. 

While  the  distrust  of  the  data  validity  is  not  unique 
to  the  LMDSS  or  NALDA  II,  it  is  a  real  as  well  as  perceived 
problem.  Additional  training  at  the  data  input  source, 
continued  development  of  NALCOMIS  Validation  Specifications, 
and  pursuit  of  Automated  Maintenance  Environment  (AME) 
initiatives  are  all  possible  means  of  improving  data 
validity. 
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A  thorough  understanding  of  the  origin  and  derivation 
of  data  is  necessary  if  users  are  to  trust  and  fully  utilize 
the  data  resource  (Brackett,  1996) .  With  the  exception  of 
the  Cost  Analysis  module,  the  algorithms  for  deriving  the 
data  and  the  data  sources  are  not  explicit.  There  is  an  on¬ 
line  data  dictionary  available,  but  in  our  opinion,  it  adds 
little  or  no  clarity  to  the  subject. 

The  lack  of  consistency  between  outputs  under  NALDA  I 
and  LMDSS/NALDA  II  should  also  be  explained  to  the  users. 
Preprocessing,  NALCOMIS  Validation  Specifications,  and  the 
use  of  Most  Probable  Logic  have  all  improved  data  accuracy, 
but  in  doing  so  created  differences  between  outputs.  The 
sources  of  these  differences  need  to  be  explicit  to  the 
user. 

When  people  understand  the  content  and  meaning  of  all 
data,  the  use  of  those  data  to  support  current  and  future 
decision  needs  is  limited  only  by  people's  imagination. 
Improve  the  quality  of  information  relative  to  timing, 
accuracy,  relevancy,  objectivity  and  understandability  and 
the  quality  of  the  resultant  decision  making  should  be 
improved.  (Stephenson,  1986;  Gatian,  1994) 


C.  LOGISTICS  CONCERNS 


The  logistics  managers  identified  the  following  as  the 
logistics  concerns  they  work  with  most  frequently  (five  or 
more  respondents  work  with  them  weekly  or  daily) . 

•  Data,  reports  requirements  -  87% 

•  Supply  support,  spares  -  87% 

•  Maintenance  planning  -  63% 

•  Readiness  degraders  -  63% 

•  Configuration  Management  -  63% 

•  Support  equipment  -  63% 

Technical  data  is  one  of  the  four  areas  identified  to  reduce 
costs  as  presented  in  the  Affordable  Readiness  Proposed 
Metrics.  Spares  and  support  equipment  are  elements  of 
inventory,  which  is  identified  as  another  area  to  reduce 
costs.  Readiness  degrader  analysis  is  central  to 
reliability-based  logistics  and  trigger-based  item 
management  that  are  used  to  implement  the  sustainment 
element  of  Affordable  Readiness,  (see  Appendix  A) 

Supply  support,  configuration  management,  support 
equipment,  maintenance  planning,  and  level  of  repair 
analysis  are  all  specifically  addressed  as  elements  of  total 
cost  of  ownership,  maintenance  concept,  standardization,  and 
supportability .  These  four  factors  are  identified  by 
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ASN(RD&A)  as  those  that  must  be  considered  to  sustain 
support  and  reduce  costs.  (see  Appendix  C) 

Logistics  managers  identified  the  following  logistic 
concerns  as  those  they  work  with  least  frequently  (five  or 
more  respondents  never  or  infrequently  work  with  them) . 

•  Facilities,  location  -  75% 

•  Facilities,  requirements  -  63% 

•  Transportation,  packaging  or  storage  -  63% 

•  Failure  mode,  effects  and  criticality  analysis  -  63% 

•  Human  factors  concerns  -  63% 

Intuitively,  the  location  of  facilities  is  of  little  concern 
to  logistics  managers  because  there  is  little  opportunity  to 
influence  this  decision.  Examining  the  influences 
surrounding  military  base  closures  and  realignments  gives  us 
a  context  in  which  to  appreciate  the  limitations  of  an  input 
by  an  individual  APML  or  PMA  to  influence  this  factor. 
Additionally,  we  understand  why  logistics  managers 
infrequently  work  with  facility  requirements  because  this  is 
only  a  concern  during  system  acquisition,  upgrade,  or 
relocation.  It  is  not  a  concern  that  impacts  every  stage  of 
the  life  cycle,  as  other  concerns  do. 

Specific  program  transportation  and  handling 
requirements  are  derived  from  the  maintenance  concept  and 
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standardization  factors  (Blanchard,  1992) .  The  Navy  has  a 
mature  and  responsive  transportation,  distribution  and 
storage  system.  Logistics  managers  are  more  concerned  with 
the  maintenance  concept  concerns  (maintenance  planning, 
level  of  repair  analysis,  local  repair  versus  transport  to 
repair  and  return)  and  standardization  concerns 
(configuration  management  and  interchangeability)  that 
contribute  to  transportation,  packaging  and  storage 
concerns.  The  Navy  transport  and  storage  system  imposes 
constraints  (weight  limits,  cubic  space  limits,  hoist  point 
or  shock  requirements)  within  which  the  APML  must  work.  We 
understand  why  a  logistics  manager  will  focus  on  the 
contributing  concerns  rather  than  fixed  constraints. 

Failure  mode,  effects,  and  criticality  analysis  (FMECA) 
is  a  design  tool  and  analysis  method  used  to  tailor  the 
complexity  of  the  design,  identify  possible  system  failures, 
the  causes  of  these  failures,  the  effects  of  the  failure  on 
the  system,  and  the  criticality  in  terms  of  safety  and_ 
mission  accomplishment  (Blanchard,  1992) .  A  logistician  who 
is  involved  with  the  early  development  of  a  system  will  work 
with  this  concern,  but  logisticians  who  do  not  get  a  voice 
in  design  will  not,  nor  will  those  who  are  working  with 
sustaining  existing  systems. 


The  objective  of  human  factors  analysis  is  to  assure 
compatibility  between  the  system  physical  and  functional 
design  features  and  the  human  element  in  the  operation, 
maintenance,  and  support  of  the  system.  Human  factor 
analysis  is  an  integral  part  of  overall  system  analysis. 
Operator  and  maintenance  personnel  requirements,  and 
training  program  needs  evolve  from  an  iterative  process  of 
evaluation,  system  modification,  and  reevaluation. 
(Blanchard,  1992)  Human  factors  concerns  are  more  directly 
associated  with  engineering  design  and  performance  analysis 
efforts.  As  logistic  concerns  become  more  integrated  with 
engineering  design  and  performance  concerns  we  can  expect 
human  factors  requirements  and  criteria  will  increase  in 
importance . 

D.  JOB  INFORMZ^TION  NEEDS 

Table  VII-1  lists  information  elements  indicated  to  be 
the  most  useful  in  performing  the  APML  job.  All  of  the 
elements  presented  in  the  questionnaire  were  useful  to  some 
degree  to  someone  and  there  was  no  element  added  by  a 
respondent.  The  elements  included  in  the  questionnaire  but 
not  'listed  in  the  table,  are  those  that  the  respondents  did 
not  consistently  indicate  as  either  slightly  useful  or 
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extremely  useful  (five  or  more  respondents;  four  or  more  for 
engine  elements) . 

The  LMDSS  provides  information  for  each  of  the 
identified  elements  except  Candidate  Identification:  Wait 
Time  Maintenance  Impact  and  Average  Days  to  Receipt  which 
are  under  development-  As  discussed  in  the  Data  Quality 
section  of  Chapter  VI,  the  only  aircraft  cost  data  for 
Maintenance  Level  3  (depot)  in  the  database  is  aggregate 
cost.  It  is  not  broken  out  by  TMS  other  than  engine 
overhaul  cost  data.  The  Reference  Information:  Production 
Load  and  Run  Statistics  element  provides  information  on  the 
LMDSS  not  aviation  programs.  Respondents  indicated  the 
element  is  useful,  but  we  believe  the  question  may  have 
falsely  led  them  to  believe  it  was  for  aviation  programs  not 
the  LMDSS.  The  respondents  did  not  use  the  LMDSS,  this 
reference  element  was  listed  among  aviation  system  reference 
elements,  and  respondents  were  not  asked  to  clarify  one 
reference  information  element  from  the  other.  Engine 
Information  elements  did  not  pertain  to  the  response 
received  from  the  Support  Equipment  APML. 

The  LMDSS  Trend  Analysis  module  is  count-based  and  does 
not  include  graphics  capabilities.  To  perform  a  regression 
analysis  or  analysis  of  possible  cause  and  effect 
relationships  the  LMDSS  data  must  be  further  manipulated 
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Summary  Data;  End  Item 
Summary  Data;  Organization 

Summary  Data;  Beyond  Capability  Maintenance  (BCM)  Report 

Reliability  Summary  Parameters 

Supportability  Summary  Parameters 

Cost  Summary  Parameters 

Emerging  Problems 

Common  Equipment 

Trend  Analysis  (Problems  and  Causes) 

Component  and/or  End  Item  Cost  Data,  specifically:  Annual  Operations  and  Support  Costs 

Component  and/or  End  Item  Cost  Data,  specifically:  Labor  Cost  History 

Component  and/or  End  Item  Cost  Data,  specifically:  Item  Value  to  Depot  Repair  Cost 

Component  and/or  End  Item  Cost  Data,  specifically:  Budget  Analysis  {OP-20  Report) 

Component  and/or  End  Item  Cost  Data,  specifically:  Inflation  Factors 

Component  and/or  Ene  Item  Cost  Data,  specifically:  Item  Value  to  Labor  Cost 

Candidate  Identification  Function,  specifically:  Wholesale  System  Demand 

Candidate  Identification  Function,  specifically:  Material  Issue  Trends 

Candidate  Identification  Function,  specifically:  Supply  Synopsis 

Candidate  Identification  Function,  specifically:  Wholesale  System  Investment 

Candidate  Identification  Function,  specifically:  Average  Customer  Wait  Reports 

Candidate  Identification  Function,  specifically:  Backorder  History  Reports 

Candidate  Identification  Function,  specifically:  NAVICP  NSN  Snapshot 

Candidate  Identification  Function,  specifically: 

Mean  Flight  Hour  Time  Between  Failure  (MTBF)  Report 
Candidate  Identification  Function,  specifically:  Wait  Time  Maintenance  Impact  * 
Candidate  Identification  Function,  specifically:  Average  Days  to  Receipt  * 

Engine  Repair  Cost 

Flight  Hours  Since  Last  Engine  Repair 
Engine  Demand  Forecasting 
Engine  Removal  Trend 

Reference  Information:  Aircraft  Inventory  and  Fatigue  Life 
Reference  Information:  Code  Definition 

Reference  Information:  Production  Load  and  Run  Statistics^* 

Reference  Information:  Possible  Courses  of  Action 
Reference  Information:  NIIN/CAGE/PN  Cross  Reference 
Reference  Information:  TEC  information 
Reference  Information:  Data  Dictionary 


Engine  information  elements  do  not  pertain  to  Support  Equipment  APML. ^ 
Aircraft  Fatigue  Life  Reference  Information  does  not  pertain  to  all  aircraft 
*  indicates  the'  LMDSS  function  is  under  development 

**  provides  reference  information  for  the  LMDSS  not  PMA  program _ 

Table  VII-1  Most  Useful  Information  Elements 


beyond  the  LMDSS  application.  We  believe  these  shortcomings 
reduce  the  utility  of  the  Trend  Analysis  function.  The 
Possible  Courses  of  Action  area  simply  includes  a  checklist 
of  actions  to  consider  and  is  not  tailored  to  a  specific 


program  or  system,  forecast,  or  available  data. 
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E.  AFFORDABLE  READINESS  DECISIONS 

The  proposed  Affordable  Readiness  metrics  discussed  in 
Chapter  II  reflect  efforts  to  more  accurately  capture  the 
total  cost  of  ownership  of  aviation  systems.  As  we  have 
discussed,  cost  reductions  are  considered  in  conjunction 
with  support,  readiness  and  safety  considerations.  The 
LMDSS  is  designed  to  be  a  tool  to  facilitate  continuous 
action  by  NAVAIR  logistics  management  teams  to  measurable ■ 
reduce  the  life  cycle  support  costs  (or  the  total  cost  of 
ownership)  of  aviation  systems  while  protecting  readiness. 

To  measurably  reduce  the  associated  costs,  the  LMDSS  must  be 
able  to  measure  the  associated  costs.  The  metrics  proposed 
by  Affordable  Readiness  define  the  required  measurements. 

The  current  architecture  and  capability  of  the  LMDSS  as  a 
NALDA  Phase  II  application  adequately  support  measurement  of 
the  metrics  associated  with  all  proposed  areas  of  Affordable 
Readiness  except  the  metrics  associated  with  safety  (Class  A 
mishaps) . 

The  reduction  in  the  life  cycle  support  costs  of 
aviation  systems  will  take  more  than  the  ability  to  measure 
associated  costs.  In  addition  to  knowing  what  the  costs  are 
one  must  be  able  to  analyze  why  and  have  incentives  to  make 
the  "right"  decisions.  The  LMDSS  has  the  ability  to  provide 
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useful  data  as  defined  by  Affordable  Readiness  metrics.  In 
addition  to  providing  data,  users  must  have  confidence  in 
those  data.  Our  research  did  not  include  a  comparison  of  the 
LMDSS  analysis  capabilities  versus  alternative  methods  of 
analyzing  data.  The  perception  of  APMLs  is  the  LMDSS  is 
good  at  "big  picture"  and  indicating  "where  to  go  look"  but 
falls  short  of  communicating  details  with  ease  or  indicating 
"why"  a  system  measurement  is  as  it  is  (such  as  what  is 
degrading  mission  capability  or  why  a  component  is  failing) . 

Naming  an  application  a  DSS  creates  certain 
expectations.  One  of  those  expectations  is  that  the  DSS 
will  support  all  three  phases  -  intelligence,  design  and 
choice  -  of  the  decision  making  process.  In  order  for  a  DSS 
to  support  the  intelligence  phase,  it  must  provide  accurate, 
timely  information.  The  design  phase  includes  inventing, 
developing  and  analyzing  possible  courses  of  action.  The 
analysis  capability  is  fulfilled  by  being  able  to  answer 
"what  if"  questions.  The  ability  to  suggest  new 
alternatives  is  met  by  being  able  to  perform  goal  seeking. 
The  choice  phase  involves  assistance  in  the  selection  of  the 
alternative  to  be  implemented.  Generally,  this  is 
accomplished  with  an  optimization  routine. 

The  LMDSS  fully  supports  the  intelligence  phase  of  the 
decision  making  process.  In  order  to  support  the  other 
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phases  of  the  decision  making  process,  the  data  must  be 
exported  to  other  applications  that  offer  models  or 
forecasting  utilities.  As  one  survey  respondent  commented, 
"The  LMDSS  can  help  answer  the  what,  but  it  can't  help  me 
with  the  why  or  the  how." 

The  LMDSS  provides  the  facility  to  export  data  to 
other  applications.  But,  if  the  LMDSS  is  to  fulfill  its 
stated  purposes  of  providing  a  repeatable  decision  making 
process  it  should  offer  a  standardized  set  of  modeling  and 
forecasting  tools  as  part  of  the  LMDSS  application. 

F.  PROGRl^  MANAGEMENT  ENVIRONMENT 

Because  the  LMDSS  will  not  be  introduced  and  cannot  be 
evaluated  in  isolation,  we  briefly  address  the  program 
management  environment  and  culture  to  more  fully  complete 
the  context  of  our  analysis. 

The  LMDSS  is  designed  to  support  APMLs  and  in  turn  to 
benefit  PMAs.  Navy  acquisition  and  program  management  is 
tied  closely  to  the  planning,  programming,  and  budgeting 
(PPBS)  process  which  determines  which  DoD  requirements  get 
funded  and  which  do  not.  Those  programs  that  get  funding 
survive.  Those  that  do  not  perish.  A  GAO  report  of 
December  1996  made  the  following  recommendations  to  improve 
opportunities  to  enhance  DoD's  Logistics  Strategic  Plan: 
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To  build  on  DoD’s  existing  strategic  planning  efforts  and  to  have  a  better 
chance  of  achieving  the  major  logistics  system  improvements  that  its  plan 
envisions,  we  recommend  Aat  the  Secretary  of  Defense  direct  the  Deputy 
Under  Secretary  for  Logistics  to  (1)  ensure  that  future  logistics  plans 
include  a  recognition  of  the  magnitude  of  the  investment  that  is  required  to 
accomplish  the  plan’s  goals,  objectives,  and  strategies  and  (2)  issue 
guidance  to  the  Secretaries  of  the  Army,  the  Navy,  and  the  Air  Force  and 
the  Director  of  DLA  instructing  the  services  and  DLA  on  how  to  link  their 
goals  and  budgets  to  the  DoD  logistic  strategic  plan’s  overall  goals  and 
strategies.  (GAO/NSIAD-97-28, 1996) 

As  indicated,  change  to  the  logistics  processes  must 
compliment  the  organizational  structure  (i.e.  be  linked  to 
overarching  Navy  goals)  and  adequate  resources  must  be 
dedicated  to  turn  strategy  to  action.  The  change  must  also 
consider  other  organizational  factors  such  as  measurements, 
control  processes,  and  reward  systems.  Acquisition  Reform 
initiatives  direct  PMAs  to  tailor  programs,  be  more 
creative,  and  to  consider  the  total  cost  of  ownership 
(DoDInst  5000.2;  Hickok,  1997;  Fox  1997).  Contrary  to  these 
directives,  the  predominant  focus  of  program  management 
remains  on  unit  cost,  schedule,  and  design  performance  (Fox, 
1997;  Eaton,  1997)  .  Incentives  continue  to  support  driving 
down  acquisition  cost  (unit  cost) ,  ensuring  timely  delivery 
of  aviation  systems  (schedule) ,  and  meeting  performance 
specifications.  To  measurably  reduce  life-cycle  support 
costs  PMAs  need  more  than  a  tool  with  which  to  measure  them. 
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As  Kaminski,  then  USD(A&T),  said  when  questioned  what  he  saw 
as  the  major  improvements  yet  to  be  achieved  in  acquisition 
reform: 

Probably  the  biggest  one  is  really  being  serious  about  addressing  life-cycle 
cost  That  is  an  area  that  I  think  we  still  talk  about  today,  but  I  do  not 
think  we  have  followed  through  with  serious  initiatives.  I  still  do  not 
believe  we  have  sufficient  incentives  in  place  for  most  program  managers 
to  seriously  consider  the  life-cycle  costs  of  their  program. .  .The  incentives 
are  still  too  much  in  the  direction  of  saving  near-year  monies,  and  that 
support  costs  will  be  someone  else’s  problem  in  the  out-years.  (Fox,  1997) 

Kaminski  tied  incentive  problems  to  the  budget  process. 
A  program  manager  has  to  put  up  near-year  funds  (taken  from 
another  program  areas)  to  make  improvements  and  then  when 
the  out-year  savings  are  realized  those  funds  are  swept  away 
and  are  not  available  to  the  program.  (Fox,  1997) 

A  logistics  analyst  for  the  S-3  aircraft  annually 
updates  a  list  of  Logistics  Engineering  Change  Proposals 
(LECPs)  that  has  initiatives  from  ten  years  ago.  The  list 
documents  projected  total  cost  savings  to  by  proposed 
investments  in  engineering  changes.  The  list  is  kept  from 
year  to  year  because  the  proposals  remain  unfunded.  Scarce 
O&S  funds  continue  to  be  spent  to  maintain  systems  that 
cannot  be  upgraded  without  investment  that  require 
procurement  funds.  The  LMDSS  may  help  identify  and  justify 
LECPs,  but  it  will  not  remedy  these  types  of  problems 
driving  up  life-cycle  costs. 
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Additionally,  while  PMAs  are  encouraged  to  be  creative, 
they  are  discouraged  from  taking  risks.  In  fact,  they  are 
expected  to  plan  carefully  to  manage  and  mitigate  risk 
(DoDInst  5000.2;  Conrow  and  Fredrickson,  1996;  Rudwick, 

1992) .  DoD/DoN  strategies  recognize  the  need  to  change 
culture  and  well  as  impediments  to  do  so  (Fox,  1997;  Hickok, 
1997;  GAO/AIMD-96-109,  1996;  GAO/NSIAD-95-28,  1994; 
GAO/AIMD/NSIAD-94-101) .  The  LMDSS  implementation  must 
consider  DoD/DoN  strategies,  environment,  culture,  and  other 
organizational  factors. 
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VIII.  CONCLUSIONS  AND  RECOMMENDATIONS 


The  data  and  information  collected  for  this  study  on  the 
effectiveness  of  the  LMDSS  to  support  logistic  management 
decision-making  provides  ample  material  to  draw  conclusions 
pertinent  to  this  study  and  identify  areas  that  warrant 
further  research. 

A.  CONCLUSIONS 

1.  The  LMDSS  is  not  a  Decision  Support  Syst^. 

A  DSS  is  composed  of  the  following  three  interrelated 
components:  data  management,  dialog  management  and  model 
management.  The  LMDSS  fully  meets  the  data  management 
component  criteria.  It  meets,  to  some  degree,  all  of  the 
dialog  management  component  criteria.  The  LMDSS  has  no 
modeling  or  sensitivity  analysis  capability.  The  LMDSS 
Trend  Analysis  module  provides  historical  data  in  tabular 
format.  Historical  data  is  presented  in  a  format  that  could 
support  time-series  forecasting,  but  not  causal,  and  there 
is  limited  "what  if"  analysis  capability.  It  is  a 
relational  database  that  improves  data  accessibility. 

2.  There  are  multiple  user  groups  who  will  be  users 

of  the  UtDSS. 
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User  groups  include  IPT  data  analysts,  FST  data 
analysts,  and  Type  Convmand  data  analysts.  These  analysts 
may  be  Navy  data  analysts,  government  employees  or  civilian 
contractors  assigned  within  the  teams. 

3.  The  LMDSS  meets  information  needs  to  in^jlement 
Affordable  Readiness  initiatives. 

The  current  architecture  and  capabilities  of  the  LMDSS 
provide  information  and  statistics  associated  with  all 
proposed  logistics  management  areas  of  Affordable  Readiness. 
No  additional  information  needs  were  identified  by  surveyed 
respondents.  Lack  of  graphics,  modeling  and  sensitivity 
analysis  capabilities  limit  identification,  analysis  and 
comparison  of  Affordable  Readiness  initiatives. 

4.  Data  quality  is  both  a  real  and  perceived  problem. 

We  identified  the  following  three  data  quality  issues: 

accessibility,  consistency  and  validity.  The  LMDSS  improves 
the  accessibility  of  data  with  the  IDE.  However,  a 
perception  exists  that  it  will  hamper  or  eliminate  the 
analyst's  ability  to  access  detail  data.  Data  consistency 
is  adequately  addressed  through  the  LMDSS  Quality  Assurance 
process.  Poor  documentation  at  the  source  degrades  data 
validity  and  the  lack  of  Maintenance  Level  3  (depot)  cost 
data  precludes  total  cost  visibility. 
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5.  The  lAlDSS  effectiveness  in  measurably  reducing 
life-cycle  support  costs  is  hampered  by  the  environment  in 
^jliich  aviation  program  manag®nent  decisions  are  made . 

The  LMDSS  has  the  capability  to  support  decisions  to 
reduce  life-cycle  support  costs,  but  PMAs  need  more  than  a 
tool  with  which  to  measure  life-cycle  costs  to  reduce  them. 
Incentives  continue  to  support,  driving  down  acquisition  cost 
(unit  cost) ,  ensuring  timely  delivery  of  aviation  systems 
(schedule),  and  meeting  performance  specifications.  The 
current  environment  encourages  short-term  decisions  that 
compromise  life-cycle  decisions .  The  LMDSS  can  help 
identify  and  justify  decisions  to  reduce  life-cycle  costs, 
but  other  factors  are  driving  up  these  same  costs. 

B .  RECOMMENDATIONS 

1.  Incorporate  a  standardized  set  of  modeling  tools 
and  sensitivity  analysis  as  part  of  the  IMDSS  application. 

To  fully  support  the  decision-making  process,  modeling 
capabilities  are  necessary.  Providing  a  standardized  set  of 
modeling  tools  will  ensure  comparable  analysis  and 
comparison  across  aviation  systems.  Sensitivity  analysis 
capabilities  would  allow  analysts  to  more  readily  assess  the 
impact  of  different  decisions. 
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2.  Incorpora'te  graphics  capabili'^  as  part  of  the 
LMDSS  application. 

Currently,  data  output  is  available  in  tabular  format 
only.  A  DSS  should  support  multiple  methods  of  presenting 
output.  This  would  add  flexibility  to  support  different 
users'  knowledge  bases. 

3 .  Enhance  availability  of  algorithm  and  data  source/ 
svimmarization  documentation. 

A  thorough  understanding  of  the  origin  and  derivation 
of  data  is  necessary  if  users  are  to  trust  and  fully  use  the 
data  resource.  Adding  specific  data  source  and  algorithm 
information  to  the  data  dictionary  is  warranted. 

4.  E3q>edite  initiatives  to  in^rove  data  validity. 

Data  validity  problems  are  not  unique  to  the  LMDSS  and 

NALDA  II.  The  Most  Probable  Logic  function  used  in  the  data 
summarization  improves  the  validity  of  summarized  data,  but 
poor  documentation  at  the  source  precludes  valid  detail 
data.  Initiatives,  such  as  Automated  Maintenance 
Environment  (AME)  and  Optimized  NALCOMIS  are  crucial  to 
meaningful  improvements  in  data  quality. 

5.  Collect  and  provide  Maintenance  Level  3  (depot) 

detail  cost  data. 
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Lack  of  detail  cost  data  from  ML  3  precludes  total  cost 
visibility.  Total  Cost  visibility  is  fundamental  to  making 
intelligent  life-cycle  cost  decisions.  Although  placeholders 
exist  in  the  LMDSS  database,  they  cannot  be  used  until  ML  3 
collects  and  provides  this  data. 

6.  Align  Budget  Process,  Reward  Structure,  and 

Strategic  Decision  efforts  to  support  life-cycle  cost 
reduction  initiatives. 

Until  the  entire  decision-making  environment  is  aligned 
around  a  common  goal  of  reducing  life— cycle  costs,  efforts 
in  this  area  will  be  fragmented  and  undermined  by  short-term 
imperatives.  Program  Managers  must  be  effective  advocates 
of  total  cost  of  ownership.  In  order  to  accomplish  this, 
they  must  be  encouraged  to  take  risks  and  be  creative  when 
considering  life  cycle  costs  and  they  must  be  rewarded  for 
doing  so. 

C.  RECOMMENDATIONS  FOR  FURTHER  RESEARCH 

1.  Evaluate  modeling  tools  currently  being  used  by 
logistics  management  teams  and  commercial  modeling  tools 
currently  available. 

A  comparison,  analysis,  and  identification  of  the  best 
set  of  standardized  modeling  tools  will  benefit  efforts  to 
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incorporate  modeling  capabilities  to  meet  logistics 
management  decision-making  needs. 

2 .  Evaluate  graphics  capabilities  currently  being 

used  by  logistics  management  teams  and  commercial  graphics 
tools  currently  available. 

A  comparison,  analysis,  and  identification  of  the  best 
set  of  graphics  tools  will  benefit  efforts  to  incorporate 
modeling  capabilities  to  meet  logistics  management  decision¬ 
making  needs. 

3.  Conduct  a  survey  of  the  newly  identified  users 
once  the  IM>SS  is  a  production  system. 

We  selected  the  APMLs  as  targets  for  our  study. 
Additional  users  were  identified.  Because  the  tasks  and 
needs  of  different  user  groups  vary,  the  information  derived 
from  this  study  of  APMLs  may  not  adequately  transfer.  Our 
study  was  also  constrained  by  the  fact  that  the  LMDSS  is 
still  a  prototype  system.  Evaluating  the  capability  of  the 
production  system  to  meet  user  needs  is  warranted. 

4.  Conduct  a  study  of  data  validity. 

During  the  course  of  this  study  we  identified  some 
areas  where  data  validity  problems  exist.  Further  research 
is  warranted  to  analyze  additional  data  validity  problem 
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areas,  assess  the  impact,  and  evaluate  alternative  courses 
of  action. 

5.  Assess  the  read±ness  for  change  and  develop  an 
organizational  transition  plan  for  is^lementing  total  cost 
of  ownership  initiatives. 

NAVAIR  is  currently  attempting  to  change  the  focus  from 
readiness  at  any  cost  to  Affordable  Readiness.  Effective 
transition  from  one  state  to  another  is  unlikely  unless 
there  is  an  adequate  perceived  need  for  change,  the 
organization  structure,  reward  system  and  processes  are  in 
place  to  support  that  change,  and  the  change-  is  effectively 
managed.  A  study  of  where  NAVAIR  is  in  the  process,  where 
they  are  going  and  how  best  to  get  there  is  warranted. 
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APPENDIX  A:  AFFORDABLE  READINESS 


Affordable  Readiness  is  a  business  practice  with  four  inter¬ 
related  elements:  flexible  sustainment,  sustained 
maintenance  planning,  rightsourcing,  and  total  cost  of 
ownership. 

Flexible  sustainment  encourages  program  managers  to  use 
performance-based  specifications;  develop  innovative,  cost 
effective,  life— cycle  solutions;  conduct  supportability 
analyses;  and  improve  reliability.  It  is  implemented 
through  reliability-based  logistics  and  trigger-based  item 
management . 

Sustained  maintenance  planning  initiatives  include 
reliability  improvements;  cycle  time  reductions;  process 
improvements;  technology  insertions;  and  infrastructure 
improvements. 

Rightsourcing  is  defined  as  "selecting  the  most 
advantageous  source  to  accomplish  a  specific  function  for  a 
weapon  system  in  its  life  cycle.  Selection  criteria 
include,  but  are  not  limited  to  life  cycle  cost,  quality, 
reliability,  safety,  and  effect  on  other  programs.  Specific 
functions  may  include  all  facets  of  Design,  Production, 
Operation,  Logistics  Support,  and  Disposal  of  the  system." 
(NAVAIR,  1998) 

Total  ownership  costs  include  all  costs  associated  with 
the  research,  development,  procurement,  operation, 
logistical  support  and  disposal  of  an  individual  weapon 
system  and  the  related  infrastructure. 

For  additional  readings  on  Affordable  Readiness, 
reliability-based  logistics,  and  trigger-based  management 
see  www.nalda.navy.mil,  NAVAIR  Logistics,  Affordable 
Readiness  Link. 
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APPENDIX  B:  INTEGRATED  PRODUCT  TEAM 


In  the  context  of  a  DoD  acquisition  program  there  are  three 
types  of  IPTs:  overarching  IPT,  working-level  IPT  and 
Program  IPT. 

The  overarching  IPT  is  formed  for  each  program  to 
provide  assistance,  and  oversight  as  the  program  proceeds 
through  the  acquisition  life  cycle.  It  is  composed  of  the 
PMA,  Program  Executive  Officer  (PEO) ,  and  appropriate 
component  staff,  joint  staff  and  Office  of  the  Secretary  of 
Defense  staff  principals  or  their  representatives. 

Working-level  IPTs  are  composed  of  the  PMA  or  his 
representative,  and  the  appropriate  staff  members  who  can 
assist  the  program  by  providing  functional  knowledge  and 
expertise  to  the  program.  For  major  programs  working-level 
IPTs  are  generally  focused  on  a  particular  discipline  or 
functional  area  such  as  supportability,  testing, 
cost /performance  or  contracting.  For  smaller  projects  one 
working-level  IPT  may  be  focused  on  the  entire  effort.  The 
integrated  IPT  is  an  exception  to  this  rule.  The  PMA  may 
establish  an  integrated  IPT  to  coordinate  the  activities  of 
the  other  working— level  IPTs.  Ideally,  the  integrating  IPT 
has  as  part  of  its  membership  one  representative  from  each 
of  the  working— level  IPTs  who  act  as  a  linking  pin  with  his 
own  working-level  IPT.  Even  though  these  teams  are  focused 
on  a  particular  functional  area,  they  are  still  multi¬ 
disciplinary.  The  supportability  IPT  should  not  be  a  team 
solely  of  logisticians  but  should  have  representatives  from 
the  disciplines  that  will  influence  the  supportability  of 
the  item. 

Program  IPTs  are  formed  at  the  program  level  to  manage  and 
execute  programs. 
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APPENDIX  C:  FACTORS  TO  SUSTAIN  SUPPORT 

AND  REDUCE  COSTS 


In  accordance  with  ASN  (RD&A)  memorandum  of  14  February 
1996,  the  following  four  factors  must  be  considered  by  Navy 
PMAs  and  their  IPTs,  Program  Executive  Officers  (PEOs) , 
Direct  Reporting  Program  Managers  (DRPMs) ,  NAVAIR  Systems 
Commanders,  and  the  Navy  Secretariat  staff  in  establishing 
supportability  requirements:  total  cost  of  ownership, 
maintenance  concept,  standardization,  and  supportability. 

An  accurate  picture  of  the  total  cost  of  ownership  and 
cost  relationships  is  necessary  for  cost  reductions.  Total 
cost  of  ownership  includes  all  costs  associated  with  the 
research,  development,  procurement,  operation,  logistical 
support  and  disposal  of  an  individual  weapons  system.  It 
includes  the  total  support  infrastructure  that  plans, 
manages,  and  executes  the  weapons  system  over  its  full  life. 
Currently,  decisions  focus  on  a  specific  cost  element, 
budget  line,  or  product  line  without  considering  the  impact 
on  the  rest  of  the  infrastructure.  For  example  savings  in 
depot  maintenance  may  increase  the  number  of  systems 
required  in  the  pipeline  to  maintain  adequate  resources  at 
the  operational  level.  Similarly,  design  changes  may 
marginally  improve  performance  but  dramatically  drive  up 
support  equipment  costs. 

The  maintenance  concept  expresses  the  strategy  for 
maintaining  the  platform  and  system  at  a  defined  level  of 
readiness  in  support  of  the  operational  scenario.  It 
includes  preventive  maintenance,  corrective  maintenance,  and 
overhaul.  Maintenance  concepts  for  the  platform,  systems, 
and  support  equipment  must  consider  maintainability  at  all 
maintenance  levels  and  must  be  consistent. 

S'banda.irdiza'bion  is  intended  to  ensure  the  minimal 
variety  and  optimal  interchangeability  of  technical 
information,  training,  equipment  parts,  and  components. 
Achieving  standardization  is  often  in  direct  opposition  to 
the  use  of  performance  specifications  and  commercial  or 
nondevelopmental  items .  A  balance  between  these  two  ends  of 
the  spectrum  is  obtained  by  using  business  and  technical 
judgement  in  determining  how  to  reduce  the  total  cost  of 
ownership. 
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Suppoir^sbxliil^y  recjuirsiuents  inust  fully  consider  life 
cycle  costs  including  possible  short  life  spans  resulting 
from  technology  insertions  or  obsolescence.  Requirements 
must  also  consider  the  risk  of  service  period  extensions. 
Planning  must  include  the  post  production  phase.  PMAs  must 
identify  the  most  cost  effective  approach  to  supporting  the 
system  when  fielded  and  assure  that  the  required  support 
elements,  data,  and  information  are  developed  and  acquired. 
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APPENDIX  D:  SUPPORTABILITY  STRATEGY 

STEPS 


"Contracting  for  Supportability"  (NAVAIR,  1998)  identifies 
five  steps  to  be  used  to  establish  a  supportability  strategy 
for  acquisition  programs  for  new  systems,  major  and  minor 
modifications  or  upgrades,  and  commercial  and 
nondevelopmental  items.  The  following  steps  should  be 
tailored  for  each  type  of  acquisition  program. 

1 .  Develop  Strategy  and  Initial  Support  Reguirenaents 

The  APML' s  first  action  is  to  determine  the  acquisition 
logistics  strategy  consistent  with  the  overall  program 
acquisition  strategy.  Major  considerations  in  determining 
the  acquisition  logistics  strategy  are  the  type  of 
acquisition,  system  complexity,  acquisition  phase, 
availability  of  historic  data,  and  time  and  resources 
available.  The  availability,  accuracy,  and  relevance  of 
experience  and  historical  databases  on  similar  existing 
systems  are  crucial  for  accomplishment  of  some  tasks. 
Available  databases  must  be  examined  to  determine  if 
extensive  work  is  needed  to  provide  focus  or  relevancy.  The 
acquisition  logistics  strategy  should  be  periodically 
reviewed  and  updated  to  reflect  any  changes  to  the  program. 
After  -the  -initial  requirements  are  selected,  further 
refinement  is  needed  to  concentrate  effort  in  high  leverage 
areas.  Specific  models  and  associated  databases  may  be 
considered  and  identified  at  this  time. 

2.  Design  Interface  with  Interrelated  Efforts 

The  APML  must  plan  how  to  interface  logistics 
requirements  with  the  engineering  community. _  Key  related 
programs  include  reliability  engineering,  maintainability 
engineering,  value  engineering,  human  systems  integration, 
system  safety  engineering,  and  transportability  engineering. 
The  acquisition  logistics  program  is  integrated  with  these 
related  programs  to  prevent  duplication  of  analyses  and  data 
and, to  ensure  that  analyses  are  performed  in  a  timely 
manner.  Logistics  data  is  sometimes  based  on,  and  should  be 
traceable  to,  systems  engineering  activities.  Design  and 
performance  information  can  be  captured,  disseminated, ^  and 
formally  controlled  to  serve  as  an  audit  trail  for  logistics 
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support  planning,  trade-off  analyses,  and  documentation 
preparation. 

3.  Select  Logistics  Products  to  be  Developed  and 
Delivered 

The  APML  must  determine  what  acquisition  logistics 
products  are  to  be  delivered  and  how  they  will  be  delivered 
(magnetic  tape,  disk,  hard  copy) .  The  importance  of 
acquiring  the  appropriate  data  must  be  emphasized  in  keeping 
with  the  evolving  policies  regarding  specifications  and 
standards  reform  and  with  the  thrust  to  reduce  data 
requirements.  The  right  data  can  be  critical.  Unnecessary 
data  is  simply  wasteful. 

4.  Determine  Supportability  Costs 

After  the  APML  has  developed  all  tasks  and  data 
selection  has  been  completed,  he  must  determine  the  costs 
associated  with  the  effort  and  document  funding 
requirements . 

5.  Finalize  Acquisition  Logistics  Strategy  and 
Document  in  Acquisition  Logistics  Plan  and 
Statement  of  Work 

These  actions  are  not  independent,  and  careful  review 
is  required  to  ensure  consistency.  After  the  acquisition 
logistics  plan  becomes  part  of  the  procurement  request  for 
the  end  item,  the  contractor  responds  with  his  support  plan. 
This  ensures  acquisition  logistics  will  be  integrated  with 
the  total  acquisition  program. 


106 


APPENDIX  E:  NALDA 


NALDA  has  been  operational  from  the  early  1980s.  It 
evolved  from  a  need  for  improved  data  analysis  capabilities 
to  support  Fleet  aviation  weapon  systems  management.  NALDA 
today  is  the  Navy  and  Marine  Corps  central  aviation 
maintenance  and  logistics  automated  information  system.  It 
provides  an  on-line,  integrated  life  cycle  logistics 
readiness  and  operational  weapons  systems  database  and  tools 
to  sustain  critical  support  analysis.  NALDA  is  accessed  and 
used  daily  by  Navy/Marine  aviation  headquarters,  fleet  and 
field  activities.  This  system  provides  accessible, 
comprehensive,  accurate  and  timely  aviation  logistics  data 
analysis  and  reporting  capabilities  to  support  fleet 
readiness,  through  sustainability  of  sophisticated  and 
complex  Naval  Aviation  weapons  and  associated  support 
equipment  and  systems.  NALDA  applications  encompass  the 
logistics  planning,  management,  administration,  budgeting, 
and  resource  allocation  in  support  of  air  weapon  systems  and 
related  support  equipment.  The  intent  of  NALDA  is  to 
support  naval  aviation  logistics  as  established  by  the  Naval 
Aviation  Maintenance  Program^  (NAVAIR,  1997) . 

A.  NALDA  Phase  I 

The  NALDA  design  has  followed  a  phased  architecture. 
Phase  I  is  currently  operational.  The  NALDA  system  is 
composed  of  hierarchical  Data  Base  Management  System  2000 
(S2K)  databases.  The  primary  source  of  data  is  the  AV3M 
data  received  via  Naval  Sea  Logistics  Command  (NSLC) . 
Secondary  sources  come  from  NADEPs  and  ASO.  It  operates  on 
the  AMDAHL  5995  mainframe  located  at  the  Defense  MegaCenter 
in  Mechanicsburg  PA.  The  telecommunications  network 
presently  consists  of  local  dial-up  and  WATS  lines. 


^  For  additional  information  on  the  Naval  Aviation 
Maintenance  Program,  see  OPNAV  4790.2G.  This  instruction 
provides  detailed  requirements  and  guidance  for  all  facets 
of  the  three  levels  of  aircraft  maintenance. 
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B.  NALDA  Phase  II 


The  current  NALDA  Phase  I  architecture  is  characterized 
by  several  proprietary  stovepipe  systems.  These  systems 
often  lack  interfaces,  and  offer  redundant  and  conflicting 
information.  Additionally,  many  of  the  applications  are 
non“Year  2000  compliant.  Phase  II  will  address  these 
deficiencies. 

Phase  I  will  be  migrated  in  two  increments  to  a 
client/server  architecture  on  the  SP2  machines  located  at 
Patuxent  River  and  employing  the  ORACLE  RDBMS.  Increment  A 
is  in  work  with  plans  to  bring  it  on  line  30  June  1998. 

This  discussion  will  focus  on  Increment  A,  as  this  is  where 
the  LMDSS  capability  is  introduced.  NALDA  II  users  will^ 
establish  a  link  to  a  NALDA  Web  Page  via  the  Internet  using 
commercial  Web  browsers  and  telecommunications  software. 

NALDA  II  provides  an  Integrated  Data  Environment  (IDE) 
which  will  include  the  functionality  of  the  systems  from 
Phase  I  with  expanded  capabilities  and  incorporated  into  new 
systems.  The  goal  is  to  create  and  store  data  once  and  use 
it  many  times.  Phase  II  will  include  the  following:  1)  a 
Logistics  Support  Analysis  Record;  2)  an  accurate 
Configuration  Management  Information  System/ Joint  Logistics 
Systems  Center  software  for  aviation  weapons  systems  - 
configuration  management  is  considered  to  be  one  of  the 
fleet's  priorities  to  improve  readiness  and  safety  of 
flight;  3)  the  LMDSS,  the  Navy's  primary  decision  support 
system  to  achieve  cost-effective  logistics  management,  more 
timely  (daily)  receipt  of  fleet  AV3M  and  configuration  data, 
cost-effective  consolidation  of  central,  upline  AV3M  data 
systems,  and  the  ability  to  access  centralized  fleet¬ 
wide,  near  real-time,  operational/readiness  data  from  NALDA; 
4)  Aircraft  Inventory  and  Readiness  Reporting  System 
(AIRRS) ;  5)  Reliability  Centered  Maintenance  (RCM) ;  6) 
Visibility  and  Management  of  Operating/Support  Cost  Programs 
(VAMOSC) ;  7)  Technical  Data  including  Joint  Computer  Aided 
Logistics  (JCALS)  Interface;  8)  Airborne  Weapons  Information 
System  (AWIS) ;  9)  Metrology  Automated  System  for  Uniform 
Recall  and  Reporting  (MEASURE);  10)  Affordable  Readiness 
Metrics/Total  Cost-Decision  Support  System  (TC-DSS) ;  11) 
Level  of  Repair  Analysis  (LORA);  and  12)  other  interfaces 
and  applications  as  identified  in  life  cycle  documentation. 
Ultimately,  the  IDE,  essentially  a  logistics  data  warehouse, 
will  contain  product  definition,  ILS  acquisition,  in-service 
management,  fleet  and  depot  maintenance,  analysis,  supply, 
cost^  configuration  management  status  reporting,  and  other 
data.  All  current  and  future  NALDA  applications  will  be 


108 


written  against  the  single  IDE  data  structure  (NAVAIR, 
1997)  . 
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APPENDIX  F:  QUESTIONNAIRE 


This  questionnaire  is  separated  into  four  parts.  Part  One  was  used 
as  part  of  a  telephone  interview.  Parts  Two  through  Four  were  mailed  to 
the  respondents  to  be  filled  out  and  then  returned. 


PHONE  INTRODUCTION 

Thank  you  for  your  contribution  to  the  research  project  of  Aerospace 
Maintenance  Duty  Officers  LCDR  Carolynn  Snyder  and  LCDR  Ellen  Moore.  We 
are  currently  pursuing  Master  of  Science  Degrees  in  Management  at  the 
Naval  Postgraduate  School,  Monterey.  Carolynn  is  a  student  in 
Information  Technology  and  Ellen  is  a  student  in  Material  Logistics  ^ 
Support.  We  are  analyzing  logistics  decision  support  in  Navy  aviation 
system  program  management.  The  quality  of  our  review  depends  on  your 
input . 

Specifically,  we  are  looking  at  the  LMDSS  system  designed  to  facilitate 
continuous  action  by  NAVAIR  logistics  management  teams  to  measurably 
reduce  the  life  cycle  support  costs  of  aviation  systems  while  protecting 
readiness.  As  a  NALDA  Phase  II  application,  it  incorporates _ data  from 
existing  maintenance,  flight,  cost,  and  material  data  bases  into  a 
repeatable  decision  making  process.  LMDSS  is  designed  to  enable 
logistics  managers  to  answer  the  following: 

1)  How  am  I  doing?  (performance  versus  plan) 

2)  What  are  my  current  and  future  support  cost  and  readiness 
drivers? 

3)  What  can  I  do  about  it? 

4)  How  much  will  the  solution  -cost? 

5)  What  is  the  payback  period? 

We  want  to  ensure  LMDSS  meets  your  needs.  We  intend  to  propose  how  the 
Navy  can  measure  if  LMDSS  measurably  reduces  the  life  cycle  support  costs 
of  aviation  systems  (LMDSS  objective) .  This  questionnaire  will  help  us 
answer  the  following: 

1)  Does  the  LMDSS  architecture  have  the  capability  of  satisfying 
APMLs? 

2)  What  information  does  an  APML  use  to  make  decisions?,  and 

3)  Does  LMDSS  provide  that  information? 


Ill 


date: 


PART  ONE  (Phone  response  from  LMDSS  USERS  AND  NON  USERS) 
completed  by  interviewer 

A.  Identification  information: 

1.  Name: - - - - - 

2 .  Phone : _ _ _ _ _ — - 

3.  e-mail: _ _ _ _ 

4.  Address: _ _ _ — 


5.  Job  Title:  _ 

6.  Brief  description  of  job: 


7.  Do  you  work  with  new  aviation  systems,  sustaining  existing  systems,  or 
both?  (circle) 


B.  How  often  do  you  use  the  following  tools  to  perform  your  job?  In  the 
’capacity  of  identifying  requirements,  analyzing  requirements  or  both? 


N:  never  Id:  identify  requirements 

I:  infrequently  A:  analyze  requirements 

M:  monthly  B:  both 

W:  weekly  DK:  don't  know 

D:  daily 

DK:  don't  know 


8.  Logistic  Support  Analysis 

9.  Raw  data 

10.  Model (s) 


(MIL-STD-1388)  N  I  M 

N  I  M 
N  I  M 


W  D  /  DK  //  Id  A  B  /  DK 
W  D  /  DK  //  Id  A  B  /  DK 
W  D  /  DK  //  Id  A  B  /  DK 


if  so,  which  one(s)? 


11.  Checklist 

if  so,  how  was  it  developed? 


N  I  M  W  D  /  DK  //  Id  A  B  /  DK 
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N  I  M  W  D  /  DK  //  Id  A  B  /  DK 


N  I  M  W  D  /  DK  //  Id  A  B  /  DK 


14.  Do  you  know  what  LMDSS,  Logistics  Management  Decision  Support 
System,  is?  (circle) 

Yes  No 

15.  Have  you  previously  or  do  you  currently  use  LMDSS?  (circle) 

Yes  No 

(skip  next  question  if  previous  answer  is  No) 

16.  If  you  use  LMDSS,  how  often?  Infrequently  Monthly  Weekly  Daily  Don’t  Know 

17.  a.  How  often  do  you  interface  with  project  engineer (s)? 

N  I  M  W  D  /  DK  //  Id  A  B  /  DK 

b.  What  for? 

c.  How  (e-mail,  phone,  conference,  etc.): 

18.  a.  How  often  do  you  interface  with  project  analyst (s)? 

N  I  M  W  D  /  DK  //  Id  A  B  /  DK 

b.  What  for? 

c.  How  (e-mail,  phone,  conference,  etc.) 

19.  a.  How  often  do  you  interface  with  those  who  influence  the  program 
budget? 

N  I  M  W  D  /  DK  //  Id  A  B  /  DK 

b.  Who? 

c.  What  for? 

d.  How  (e-mail,  phone,  conference,  etc.) 

20.  Other: 

a.  Who?  N  I  M  W  D/DK //  IdAB/DK 

b.  What  for? 

c.  How  (e-mail,  phone,  conference,  etc.) 

21.  How  do  you  measure  life  cycle  costs? 


12.  Intuition/experience 

13.  other: 
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This  questionnaire  has  been  developed  for  APMLs  assigned  to  aviation 
system  programs.  Do  you  know  of  anyone  else  you  feel  would  make  a 
valuable  contribution  to  our  study,  particularly  anyone  involved  wi 
logistics  processes  in  aviation  systems  program  management. 


Name : 
Phone : 
Address : 


Name: 
Phone : 
Address : 


Name : 
Phone : 
Address : 
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(MAIL)  ....  personalized  address 

Thank  you  again  for  your  contribution  to  the  research  project  of 
Aerospace  Maintenance  Duty  Officers  LCDR  Carolynn  Snyder  and  LCDR  Ellen 
Moore.  As  we  discussed  by  phone  (date),  we  are  currently  pursuing  Master 
of  Science  Degrees  in  Management  at  the  Naval  Postgraduate  School, 
Monterey.  Carolynn  is  a  student  in  Information  Technology  and  Ellen  is  a 
student  in  Material  Logistics  Support.  We  are  analyzing  logistics 
decision  support  in  Navy  aviation  system  program  management.  The  quality 
of  our  review  depends  on  your  input. 

Specifically,  we  are  looking  at  the  LMDSS  system  designed  to  facilitate 
continuous  action  by  NAVAIR  logistics  management  teams  to  measurably 
reduce  the  life  cycle  support  costs  of  aviation  systems  while  protecting 
readiness.  As  a  NALDA  Phase  II  application,  it  incorporates _ data  from 
existing  maintenance,  flight,  cost,  and  material  data  bases  into  a 
repeatable  decision  making  process.  LMDSS  is  designed  to  enable 
logistics  managers  to  answer  the  following: 

1)  How  am  I  doing?  (performance  versus  plan) 

2)  What  are  my  current  and  future  support  cost  and  readiness 
drivers? 

3)  What  can  I  do  about  it? 

4)  How  much  will  the  solution  cost? 

5)  What  is  the  payback  period? 

We  want  to  ensure  LMDSS  meets  your  needs.  We  intend  to  propose  how  the 
Navy  can  measure  if  LMDSS  measurably  reduces  the  life  cycle  support  costs 
of  aviation  systems  (LMDSS  objective) .  This  questionnaire  will  help  us 
answer  the  following: 

1)  Does  the  LMDSS  architecture  have  the  capability  of  satisfying 
APMLs? 

2)  What  information  does  an  APML  use  to  make  decisions?,  and 

3)  Does  LMDSS  provide  that  information? 


Please  feel  free  to  address  any  questions  or  comments  to: 

LCDR  Ellen  Moore/phone:  408-657-0891/email:  eemoore@nps.navy.mil. 
or  LCDR  Carolynn  Snyder/phone:  408-393-9567/email:  cmsnyder@nps.navy.mil. 

The  questionnaire  is  divided  into  four  parts: 

Part  ONE:  Information  provided  by  phone  (please  review  and  provide 
additional  comments  as  desired) . 

Part  TWO:  Written  response  from  LMDSS  users. 

Part  THREE:  Written  response  from  those  who  have  not  used  LMDSS. 

Part  FOUR:  Written  response,  job  information  needs  (LMDSS  users  and 
non-users) . 
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Please  mark  each  section  with  a  legible  pen  or  pencil.  Attach  additional 
pages  as  required. 

PART  TWO  (LMDSS  Users) 

C.  How  frequently  do  you  work  with  the  following  logistics  concerns? 
Additionally  indicate  if  it  is  in  the  capacity  of  identifying 
requirements,  analyzing  requirements  or  both,  (circle  the  best  answer) 

22.  Logistic  criteria  as  input  to  system  design 
{reliability/maintainability  goals/objectives) 

a  .frequency:  NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON'T  KNOW 

b.  capacity:  IDENTIFY  ANALYZE  BOTH  DON’T  KNOW 

REQUIREMENTS  REQUIREMENTS 


23.  Human  factors  concerns 


a  .frequency: 
b.  capacity: 


NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

IDENTIFY  ANALYZE  BOTH  DON’T  KNOW 

REQUIREMENTS  REQUIREMENTS 


24.  Failure  mode,  effects,  and  critical  analysis 


a  .frequency: 
b.  capacity: 


NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

IDENTIFY  ANALYZE  BOTH  DON’T  KNOW 

REQUIREMENTS  REQUIREMENTS 


25.  Failure  reporting,  analysis,  and  corrective-action  system 


a  .frequency: 
b.  capacity: 


NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

IDENTIFY  ANALYZE  BOTH  DON’T  KNOW 

REQUIREMENTS  REQUIREMENTS 


26.  Provisioning  needs/alternatives 


a  .frequency: 
b.  capacity: 


NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

IDENTIFY  ANALYZE  BOTH  DON’T  KNOW 

REQUIREMENTS  REQUIREMENTS 
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27.  Compatibility  with  existing  system 


a  . frequency: 
b.  capacity: 


NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

IDENTIFY  ANALYZE  BOTH  DONTKNOW 

REQUIREMENTS  REQUIREMENTS 


28.  Configuration  Management 

,  a  .frequency:  NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DONTKNOW 

b.  capacity:  IDENTIFY  ANALYZE  '  BOTH  DONTKNOW 

REQUIREMENTS  REQUIREMENTS 

29.  Training  and  training  support 

a  .frequency:  NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DONTKNOW 

b.  capacity:  IDENTIFY  ANALYZE  BOTH  DON’T  KNOW 

REQUIREMENTS  REQUIREMENTS 


30.  Manpower  and  personnel  __ 

a  .frequency:  NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

b.  capacity:  IDENTIFY  ANALYZE  BOTH  DON’T  KNOW 

REQUIREMENTS  REQUIREMENTS 

31.  Supply  support,  spares 

a  .frequency:  NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

b.  capacity:  IDENTIFY  ANALYZE  BOTH  DON’T  KNOW 

REQUIREMENTS  REQUIREMENTS 


32.  Inventory  level  analysis 

a  .frequency:  NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

b.  capacity:  IDENTIFY  ANALYZE  BOTH  DON’T  KNOW 

REQUIREMENTS  REQUIREMENTS 
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33. 


Transportation, 
a  .frequency: 
b.  capacity: 


packaging  or  storage 

NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY 

IDENTIFY  ANALYZE  BOTH 

REQUIREMENTS  REQUIREMENTS 


NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY 

IDENTIFY  ANALYZE  BOTH 

REQUIREMENTS  REQUIREMENTS 

35.  Support  equipment 

a  .frequency:  NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY 

b.  capacity:  IDENTIFY  ANALYZE  BOTH 

REQUIREMENTS  REQUIREMENTS 

36.  Computer  resource  support 

a  .frequency:  NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY 

b.  capacity:  IDENTIFY  ANALYZE  BOTH 

REQUIREMENTS  REQUIREMENTS 

37.  Facilities,  requirements 

a  .frequency:  NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY 

b.  capacity:  IDENTIFY  ANALYZE  BOTH 

REQUIREMENTS  REQUIREMENTS 

38.  Facilities,  location 

a  .frequency:  NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY 

b.  capacity:  IDENTIFY  ANALYZE  BOTH 

REQUIREMENTS  REQUIREMENTS 


34 .  Test  equipment 
a  .frequency: 
b.  capacity: 


DON’T  KNOW 
DON’T  KNOW 

DON’T  KNOW 
DON’T  KNOW 

DON’T  KNOW 
DON’T  KNOW 

DON’T  KNOW 
DON’T  KNOW 

DON’T  KNOW 
DON’T  KNOW 

DON’T  KNOW 
DON’T  KNOW 
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39.  Data,  reports  requirements 


a  .  frequency: 
b.  capacity: 


NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

IDENTIFY  ANALYZE  BOTH  DON’T  KNOW 

REQUIREMENTS  REQUIREMENTS 


40.  Maintenance  planning  (scheduled  versus  unscheduled  plan) 


a  . frequency: 
b.  capacity: 


NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

IDENTIFY  ANALYZE  BOTH  DON’T  KNOW 

REQUIREMENTS  REQUIREMENTS 


41.  Level  of  repair  analysis  (O  versus  I  versus  D-levels) 

a  .frequency:  NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON T KNOW 

b.  capacity:  IDENTIFY  ANALYZE  BOTH  DON T KNOW 

REQUIREMENTS  REQUIREMENTS 

42.  Operating  environment  issues 

NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

IDENTIFY  ANALYZE  BOTH  DON’T  KNOW 

REQUIREMENTS  REQUIREMENTS 


NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

IDENTIFY  ANALYZE  BOTH  DON’T  KNOW 

REQUIREMENTS  REQUIREMENTS 

44.  Readiness  degraders 

a  .frequency:  NEVER  INFREQUENTLY  MONtHLY  WEEKLY  DAILY  DON’T  KNOW 

b.  capacity:  IDENTIFY  ANALYZE  BOTH  DON’T  KNOW 

REQUIREMENTS  REQUIREMENTS 


a  .  frequency: 
b.  capacity: 

43.  Cost-drivers 
a  .frequency: 
b.  capacity: 
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45.  Cycle  time  to  repair  components 


a  .frequency: 
b.  capacity: 


NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

IDENTIFY  ANALYZE  BOTH  DON’T  KNOW 

REQUIREMENTS  REQUIREMENTS 


46.  other: 

a  .frequency: 
b.  capacity: 


NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

IDENTIFY  ANALYZE  BOTH  DON’T  KNOW 

REQUIREMENTS  REQUIREMENTS 


47.  How  often  do  you  use  LMDSS? 

NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 


****  If  you  do  not  use  LMDSS  then  the  next  portion  of  this  questionnaire 
has  been  sent  to  you  in  error.  STOP  NOW  and  call  LCDR  Ellen  Moore/phone 
408-657-0891  or  LCDR  Carolynn  Snyder/phone  408-393-9567  for  the  correct 
questionnaire  for  NON-USERS. 

If  you  do  use  LMDSS,  please  continue  with  the  questionnaire. 


D.  How  often  do  you  use  the  followinq  types  of  LMDSS  queries?  (circle 
the  best  answer) 

48.  Summary  data  for  end  items 

NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  '  DON'T  KNOW 

49.  Reliability  summary  parameters 

NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

50.  Supportability  summary  parameters 

NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 
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51.  Cost  summary  parameters 

NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

52.  Trend  analysis  (problems  and  causes) 

NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

53.  Component  and/or  end-item  cost  data,  specifically:  Annual  Operations 
and  Support  Costs 

NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

54.  Component  and/or  end— item  cost  data,  specifically:  Labor  Cost  History 

NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 


55.  Component  and/or  end-item  cost  data,  specifically:  Item  Value  to 
Depot  Repair  Cost 

NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

56.  Component  and/or  end-item  cost  data,  specifically:  Budget  Analysis 
(OP-20  Report) 

NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

57.  Component  and/or  end-item  cost  data,  specifically:  Inflation  Factors 


NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

58.  Component  and/or  end-item  cost  data,  specifically:  Item  Value  to 
Labor  Cost 

NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

59.  Candidate  Identification  Function,  specifically:  Detailed  Component 
Report 

NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

60.  Candidate  Identification  Function,  specifically:  Wholesale  System 
Demand 

NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 
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61.  Candidate  Identification  Function,  specifically:  Material  Issue 
Trends 


NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY 


DON’T  KNOW 


62.  Candidate  Identification  Function,  specifically:  Supply  Synopsis 


NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

63.  Candidate  Identification  Function,  specifically:  Wholesale  System 
Investment 

NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

64 .  "Candidate  Identification  Function,  specifically:  Average  Customer 
Wait  Reports 

NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

65.  Candidate  Identification  Function,  specifically:  Backorder  History 
Reports 

NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

66.  Candidate  Identification  Function,  specifically:  NAVICP  NSN  Snapshot 


NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

67.  Candidate  Identification  Function,  specifically:  Mean  Flight  Hour 
Between  Failure  Report 


NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY 

68.  Engine  Repair  Cost 

NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY 

69.  Flight  Hours  Since  Last  Engine  Repair 

NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY 

70.  Engine  Demand  Forecasting 

NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY 


DON’T  KNOW 


DON’T  KNOW 


DON’T  KNOW 


DON’T  KNOW 
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71.  Engine  Overview 

NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

72.  Reference  Information:  Aircraft  Inventory  and  Fatigue  Life 

NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

73. .  Reference  Information:  Code  Definition 

NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

74.  Reference  Information:  Aircraft  Engine  Installation  and  Ownership 


NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

75.  Reference  Information:  Production  Load  and  Run  Statistics 

NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

76.  Reference  Information:  Possible  Courses  of  Action  — 

NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

77.  Reference  Information:  Organization  Codes  /Job  Count 

never  infrequently  monthly  WEEKLY  DAILY  DON’T  KNOW 

78.  Reference  Information:  NI IN /CAGE /Part  Number  Cross  Reference 


NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY 

79.  Reference  Information:  TEC  Information 

NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY 

80.  Reference  Information:  SALTS  File  Information 

NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY 

81.  Reference  Information:  Data  Dictionary 

NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY 


DON’T  KNOW 


DON’T  KNOW 


DON’T  KNOW 


DON’T  KNOW 


123 


E.  Please  describe  your  experience  with  the  following  LMDSS  functions? 
(circle  the  best  answer) 

Please  include  comments  to  clarify  answers. 


82.  Interface 


STRONGLY 

DISLIKE 

DISLIKE 

NEUTRAL 

LIKE 

STRONGLY 

LIKE 

DON’T  KNOW 

comments: 

83.  Help  function 

STRONGLY 

DISLIKE 

DISLIKE 

NEUTRAL 

LIKE 

STRONGLY 

LIKE 

DON’T  KNOW 

comments: 

84.  Analysis 

Tools 

STRONGLY 

DISLIKE 

DISLIKE 

NEUTRAL 

LIKE 

STRONGLY 

LIKE 

DON’T  KNOW 

comments: 

85.  Report  Presentation/format 

STRONGLY 

DISLIKE 

DISLIKE 

NEUTRAL 

LIKE 

STRONGLY 

LIKE 

DON’T  KNOW 

comments: 

86.  Time  required  getting  what  is 

needed 

STRONGLY 

DISLIKE 

DISLIKE 

NEUTRAL 

LIKE 

STRONGLY 

LIKE 

DON’T  KNOW 

comments: 
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87.  Ease  of  getting  what  is  needed 

STRONGLY  DISLIKE  NEUTRAL  LIKE  STRONGLY  DON’T  KNOW 

DISLIKE  like 

comments: 


88.  Training 

STRONGLY 

DISLIKE 

DISLIKE 

NEUTRAL 

LIKE 

STRONGLY 

LIKE 

DON’T  KNOW 


comments: 


89.  Accessibility  (when  desired) 

STRONGLY  DISLIKE  NEUTRAL  LIKE  STRONGLY  DON’T  KNOW 

DISLIKE  like 


comments: 


90.  Accessibility  (server  access) 

STRONGLY  DISLIKE  NEUTRAL  LIKE  STRONGLY  DON’T  KNOW 

DISLIKE  like 


comments: 


91.  Accessibility  (password  access) 

STRONGLY  DISLIKE  NEUTRAL  LIKE  STRONGLY 
DISLIKE  like 


DON’T  KNOW 


comments: 

92.  Provides  the  information  I  need 

STRONGLY  DISLIKE  NEUTRAL  LIKE  STRONGLY  DON’T  KNOW 

DISLIKE  like 


comments: 
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F.  Please  describe  your  experience  using  the  LMDSS  data 
available,  (circle  the  best  answer) 

Please  include  comments  to  clarify  answers. 


93.  Data  meets  my  needs 

STRONGLY  DISAGREE 

DISAGREE 

comments: 

94.  Data  is  accessible 

STRONGLY  DISAGREE 

DISAGREE 


NEUTRAL  AGREE  STRONGLY 

AGREE 


NEUTRAL  AGREE  STRONGLY 

AGREE 


comments: 


95.  Data  is  accurate/consistent 

STRONGLY  DISAGREE  NEUTRAL  AGREE  STRONGLY 
DISAGREE  AGREE 

comments: 


96.  Data  is  detailed  enough 

STRONGLY  DISAGREE  NEUTRAL  AGREE  STRONGLY 
DISAGREE  AGREE 


comments: 


97.  The  exact  data  meaning  is  clear 

STRONGLY  DISAGREE  NEUTRAL  AGREE  STRONGLY 
DISAGREE  AGREE 

comments: 


currently 


DON’T  KNOW 


DON’T  KNOW 


DON’T  KNOW 


DON’T  KNOW 


DON’T  KNOW 
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G.  Please  describe  your  experience  using  other  data  currently  available, 
(circle  the  best  answer) 

Please  include  comments  to  clarify  answers. 

98.  Data  meets  my  needs 

STRONGLY  DISAGREE  NEUTRAL  AGREE  STRONGLY  DON’T  KNOW 

DISAGREE  AGREE 

comments: 


99.  Data  is  accessible 

STRONGLY  DISAGREE  NEUTRAL  AGREE  STRONGLY  DON’T  KNOW 

DISAGREE  AGREE 


comments: 


100.  Data  is  accurate/consistent 

STRONGLY  DISAGREE  NEUTRAL  AGREE  STRONGLY  DON’T  KNOW 

DISAGREE  AGREE 


comments: 


101.  Data  is  detailed  enough 

STRONGLY  DISAGREE  NEUTRAL  AGREE  STRONGLY  DON’T  KNOW 

DISAGREE  AGREE 


comments: 


102.  The  exact  data  meaning  is  clear 

STRONGLY  DISAGREE  NEUTRAL  AGREE  STRONGLY  DON’T  KNOW 

DISAGREE  AGREE 


comments: 


Please  continue  now  with  PART  FOUR,  Section  J,  Job  Information  Needs 
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Please  mark  each  section  with  a  legible  pen  or  pencil.  Attach  additional 
pages  as  required. 


PART  THREE  (LMDSS  Non-users) 

H.  How  frequently  do  you  work  with  the  following  logistics  concerns? 
Additionally  indicate  if  it  is  in  the  capacity  of  identifying 
requirements,  analyzing  requirements  or  both,  (circle  the  best  answer) 


103.  Logistic  criteria  as  input  to  system  design 
(reliability/maintainability  goals/objectives) 

a  .frequency:  NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON'T  KNOW 

b.  capacity:  IDENTIFY  ANALYZE  BOTH  DON’T  KNOW 

REQUIREMENTS  REQUIREMENTS 


104.  Human  factors  concerns 


a  .frequency: 
b.  capacity: 


NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

IDENTIFY  ANALYZE  BOTH  DON’T  KNOW 

REQUIREMENTS  REQUIREMENTS 


105.  Failure  mode,  effects,  and  critical  analysis 


a  .frequency: 
b.  capacity: 


NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

IDENTIFY  ANALYZE  BOTH  DON’T  KNOW 

REQUIREMENTS  REQUIREMENTS 


106.  Failure  reporting,  analysis,  and  corrective-action  system 


a  .frequency: 
b.  capacity: 


NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

IDENTIFY  ANALYZE  BOTH  DON’T  KNOW 

REQUIREMENTS  REQUIREMENTS 


107.  Provisioning  needs/alternatives 


a  .frequency: 
b.  capacity: 


NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

IDENTIFY  ANALYZE  BOTH  DON’T  KNOW 

REQUIREMENTS  REQUIREMENTS 
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108.  Compatibility  with  existing  system 


a  . frequency: 
b.  capacity: 


NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

IDENTIFY  ANALYZE  BOTH  DON’T  KNOW 

REQUIREMENTS  REQUIREMENTS 


109.  Configuration  Management 


a  .  frequency: 

NEVER  INFREQUENTLY 

MONTHLY  WEEKLY  DAILY 

DON’T  KNOW 

b.  capacity: 

IDENTIFY 

REQUIREMENTS 

ANALYZE  BOTH 

REQUIREMENTS 

DON’T  KNOW 

Training  and 

training  support 

■ 

a  .frequency: 

NEVER  INFREQUENTLY 

MONTHLY  WEEKLY  DAILY 

DON’T  KNOW 

b.  capacity: 

IDENTIFY 

REQUIREMENTS 

ANALYZE  BOTH 

REQUIREMENTS 

DON’T  KNOW 

111.  Manpower  and  personnel 


a  .  frequency: 
h.  capacity: 


NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

IDENTIFY  ANALYZE  BOTH  DON’T  KNOW 

REQUIREMENTS  REQUIREMENTS 


112.  Supply  support,  spares 


a  .  frequency: 
b.  capacity: 


NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

IDENTIFY  ANALYZE  BOTH  DON’T  KNOW 

REQUIREMENTS  REQUIREMENTS 


113.  Inventory  level  analysis 


a  .frequency: 
b.  capacity: 


NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

IDENTIFY  ANALYZE  BOTH  DON’T  KNOW 

REQUIREMENTS  REQUIREMENTS 
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114.  Transportation,  packaging  or  storage 


a  .frequency:  NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY 

b.  capacity:  IDENTIFY  ANALYZE  BOTH 

REQUIREMENTS  REQUIREMENTS 

115.  Test  equipment 

a  .frequency:  NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY 

b.  capacity:  IDENTIFY  ANALYZE  BOTH 

REQUIREMENTS  REQUIREMENTS 

116.  Support  equipment 

a  .frequency:  NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY 

b.  capacity:  IDENTIFY  ANALYZE  BOTH 

REQUIREMENTS  REQUIREMENTS 

117 .  Computer  resource  support 

a  .frequency:  NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY 

b.  capacity:  IDENTIFY  ANALYZE  BOTH 

REQUIREMENTS  REQUIREMENTS 

118.  Facilities,  requirements 

a  .frequency:  NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY 

b.  capacity:  IDENTIFY  ANALYZE  BOTH 

REQUIREMENTS  REQUIREMENTS 

119.  Facilities,  location 

a  .frequency:  NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY 

b.  capacity:  IDENTIFY  ANALYZE  BOTH 

REQUIREMENTS  REQUIREMENTS 


DON’T  KNOW 
DON’T  KNOW 

DON’T  KNOW 
DON’T  KNOW 

DON’T  KNOW 
DON’T  KNOW 

DON’T  KNOW 
DON’T  KNOW 

DON’T  KNOW 
DON’T  KNOW 

DON’T  KNOW 
DON’T  KNOW 
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120.  Data,  reports  requirements 


a  .frequency: 

NEVER  INFREQUENTLY 

MONTHLY  WEEKLY  DAILY 

DON’T  KNOW 

b.  capacity: 

IDENTIFY 

ANALYZE 

BOTH 

DON’T  KNOW 

REQUIREMENTS 

REQUIREMENTS 

121.  Maintenance  planning  (scheduled  versus  unscheduled  plan) 

a  .  frequency: 

NEVER  INFREQUENTLY 

MONTHLY  WEEKLY  DAILY 

DON’T  KNOW 

b.  capacity: 

IDENTIFY 

ANALYZE 

BOTH 

DON’T  KNOW 

REQUIREMENTS 

REQUIREMENTS 

122.  Level  of  repair  analysis  (0  versus  I  versus  D 

-levels) 

a  .  frequency: 

NEVER  INFREQUENTLY 

MONTHLY  WEEKLY  DAILY 

DON’T  KNOW 

b.  capacity: 

IDENTIFY 

ANALYZE 

BOTH 

DON’T  KNOW 

REQUIREMENTS 

REQUIREMENTS 

123.  Operating  environment  issues 

a  .  frequency: 

NEVER  INFREQUENTLY 

MONTHLY  WEEKLY  DAILY 

DON’T  KNOW 

b.  capacity: 

IDENTIFY 

ANALYZE 

BOTH 

DON’T  KNOW 

REQUIREMENTS 

REQUIREMENTS 

124.  Cost-drivers 

a  . frequency: 

NEVER  INFREQUENTLY 

MONTHLY  WEEKLY  DAILY 

DON’T  KNOW 

b.  capacity: 

IDENTIFY 

ANALYZE 

BOTH 

DON’T  KNOW 

REQUIREMENTS 

REQUIREMENTS 

125.  Readiness  degraders 

a  . frequency: 

NEVER  INFREQUENTLY 

MONTHLY  WEEKLY  DAILY 

DON’T  KNOW 

b.  capacity: 

IDENTIFY 

ANALYZE 

BOTH 

DON’T  KNOW 

REQUIREMENTS 

REQUIREMENTS 
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126.  Cycle  time  to  repair  components 


a  .frequency: 
b.  capacity: 

127.  other: 

a  .frequency: 
b.  capacity: 


NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

IDENTIFY  ANALYZE  BOTH  DON’T  KNOW 

REQUIREMENTS  REQUIREMENTS 


NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 

IDENTIFY  ANALYZE  BOTH  DON’T  KNOW 

REQUIREMENTS  .  REQUIREMENTS 


128.  How  often  do  you  use  LMDSS? 

NEVER  INFREQUENTLY  MONTHLY  WEEKLY  DAILY  DON’T  KNOW 


****  If  you  use  LMDSS  then  the  next  portion  of  this  questionnaire  has 
been  sent  to  you  in  error.  STOP  NOW  and  call  LCDR  Ellen  Moore /phone  408- 
657-0891  or  LCDR  Carolynn  Snyder/phone  408-393-9567  for  the  correct 
questionnaire  for  NON-USERS. 

If  you  do  not  use  LMDSS,  please  continue  with  the  questionnaire. 


129.  Why  do  you  not  use  LMDSS?  (check  mark  all  that  apply) 

_ I  didn't  know  it  existed 

My  PC  won't  support  LMDSS 

_ I've  received  no  training 

I  don't  need  the  information  LMDSS  provides 
It  takes  too  long  to  get  what  I  need 

_ It's  too  difficult  to  get  what  I  need 

It  doesn't  provide  me  the  information  I  need 
What  information  do  you  need  that  isn't  provided? 
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I.  Please  describe  your  experience  using  other  data  currently  available, 
(circle  the  best  answer) 

Please  include  coininents  to  clarify  answers. 

130.  Data  meets  my  needs 


STRONGLY 

DISAGREE 

DISAGREE  NEUTRAL 

AGREE 

STRONGLY 

AGREE 

DONT  KNOW 

comments: 

131.  Data  is 

accessible 

STRONGLY 

DISAGREE 

DISAGREE  NEUTRAL 

AGREE 

STRONGLY 

AGREE 

DON’T  KNOW 

comments: 

132.  Data  is 

accurate/ consistent 

STRONGLY 

DISAGREE 

DISAGREE  NEUTRAL 

AGREE 

STRONGLY 

AGREE 

DONT  KNOW 

comments: 

133.  Data  is 

detailed  enough 

STRONGLY 

DISAGREE 

DISAGREE  NEUTRAL 

AGREE 

STRONGLY 

AGREE 

DON’T  KNOW 

comments: 

134.  The  exact  data  meaning  is  clear 

STRONGLY 

DISAGREE 

DISAGREE  NEUTRAL 

AGREE 

STRONGLY 

AGREE 

DON’T  KNOW 

comments: 


Please  continue  now  with  PART  FOUR,  Section  J,  Job  Information  Needs 
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Please  mark  each  section  with  a  legible  pen  or  pencil.  Attach  additional 
pages  as  required. 

PART  FOtJR  (Job  Information  Needs) 

J.  Indicate  how  useful  the  following  information  elements  are  (or  would 
be)  in  performing  your  job-  If  an  element  is  not  provided,  please 
include  as  an  addition,  (circle  the  best  answer) 

135.  Summary  data;  End  Item 


NOT  AT  ALL 
USEFUL 

NOT  NEUTRAL 

USEFUL 

SLIGHTLY 

USEFUL 

EXTREMELY 

USEFUL 

DON’T  KNOW 

136.  Summary 

data;  Claimant 

NOT  AT  ALL 
USEFUL 

NOT  NEUTRAL 

USEFUL 

SLIGHTLY 

USEFUL 

EXTREMELY 

USEFUL 

DON’T  KNOW 

137.  Summary 

data;  Organization 

NOT  AT  ALL 
USEFUL 

NOT  NEUTRAL 

USEFUL 

SLIGHTLY 

USEFUL 

EXTREMELY 

USEFUL 

DON’T  KNOW 

138.  Summary 

data;  BCM  Report 

NOT  AT  ALL 
USEFUL 

NOT  NEUTRAL 

USEFUL 

SLIGHTLY 

USEFUL 

EXTREMELY 

USEFUL 

DON’T  KNOW 

139.  Reliability  summary  parameters 

NOT  AT  ALL 
USEFUL 

NOT  NEUTRAL 

USEFUL 

SLIGHTLY 

USEFUL 

EXTREMELY 

USEFUL 

DON’T  KNOW 

140.  Supportability  summary  parameters 

NOT  AT  ALL 
USEFUL 

NOT  NEUTRAL 

USEFUL 

SLIGHTLY 

USEFUL 

EXTREMELY 

USEFUL 

DON’T  KNOW 

141.  Cost  summary  parameters 

NOT  AT  ALL 
USEFUL 

NOT  NEUTRAL 

USEFUL 

SLIGHTLY 

USEFUL 

EXTREMELY 

USEFUL 

DON’T  KNOW 

142.  Emerging  Problems 

NOT  AT  ALL 
USEFUL 

NOT  NEUTRAL 

USEFUL 

SLIGHTLY 

USEFUL 

EXTREMELY 

USEFUL 

DON’T  KNOW 
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143.  Common  Equipment 


NOT  AT  ALL  NOT 

USEFUL  USEFUL 

NEUTRAL 

SLIGHTLY 

USEFUL 

EXTREMELY 

USEFUL 

DON’T  KNOW 

144.  Trend  analysis  (problems  and  causes) 

NOT  AT  ALL  NOT 

USEFUL  USEFUL 

NEUTRAL 

SLIGHTLY 

USEFUL 

EXTREMELY 

USEFUL 

DON’T  KNOW 

14  5-.  Component  and/or 
and  Support  Costs 

end-item 

cost  data. 

specifically: 

Annual  Operations 

NOT  AT  ALL  NOT 

USEFUL  USEFUL 

NEUTRAL 

SLIGHTLY 

USEFUL 

EXTREMELY 

USEFUL 

DON’T  KNOW 

146.  Component  and/or 
History 

end-item 

cost  data. 

specifically: 

Labor  Cost 

NOT  AT  ALL  NOT 

USEFUL  USEFUL 

NEUTRAL 

SLIGHTLY 

USEFUL 

EXTREMELY 

USEFUL 

DON’T  KNOW 

147.  Component  and/or 
Depot  Repair  Cost 

end- item 

cost  data. 

specifically: 

Item  Value  to  .  - 

NOT  AT  ALL  NOT 

USEFUL  USEFUL 

NEUTRAL 

SLIGHTLY 

USEFUL 

EXTREMELY 

USEFUL 

DON’T  KNOW 

148.  Component  and/or 
(OP-20  Report) 

end- item 

cost  data. 

specifically: 

Budget  Analysis 

NOT  AT  ALL  '  NOT 

USEFUL  USEFUL 

NEUTRAL 

SLIGHTLY 

USEFUL 

EXTREMELY 

USEFUL 

DON’T  KNOW 

149.  Component  and/or 

end-item 

cost  data. 

specifically: 

Inflation  Factors 

NOT  AT  ALL  NOT 

USEFUL  USEFUL 

NEUTRAL 

SLIGHTLY 

USEFUL 

EXTREMELY 

USEFUL 

DON’T  KNOW 

150.  Component  and/or 
Labor  Cost 

end-item 

cost  data. 

specifically: 

Item  Value  to 

NOT  AT  ALL  NOT 

USEFUL  USEFUL 

NEUTRAL 

SLIGHTLY 

USEFUL 

EXTREMELY 

USEFUL 

DON’T  KNOW 

151.  Candidate  Identification  Function,  specifically:  Detailed  Component 
Report 

NOT  AT  ALL  NOT  NEUTRAL  SLIGHTLY  EXTREMELY  DON’T  KNOW 

USEFUL  USEFUL  USEFUL  USEFUL 
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specif ically t  Wholesale  System 


152.  Candidate  Identification  Function, 
Demand 

NOT  AT  ALL  NOT  NEUTRAL  SLIGHTLY 

USEFUL  USEFUL  USEFUL 


EXTREMELY 

USEFUL 


153.  Candidate  Identification  Function,  specifically: 
Trends 


NOT  AT  ALL 
USEFUL 


NOT  NEUTRAL  SLIGHTLY  EXTREMELY 
USEFUL  USEFUL  USEFUL 


154.  Candidate  Identification  Function, 


specifically: 


NOT  AT  ALL  NOT  NEUTRAL  SLIGHTLY  EXTREMELY 

USEFUL  USEFUL  USEFUL  USEFUL 

155.  Candidate  Identification  Function,  specifically: 
Investment 


NOT  AT  ALL 
USEFUL 


NOT  NEUTRAL  SLIGHTLY  EXTREMELY 
USEFUL  USEFUL  USEFUL 


156.  Candidate  Identification  Function,  specifically: 
Wait  Reports 

NOT  AT  ALL  NOT  NEUTRAL  SLIGHTLY  EXTREMELY 

USEFUL  USEFUL  USEFUL  USEFUL 


157.  Candidate  Identification  Function,  specifically: 
Reports 

NOT  AT  ALL  NOt  NEUTRAL  SLIGHTLY  EXTREMELY 

USEFUL  USEFUL  USEFUL  USEFUL 


158.  Candidate  Identification  Function,  specifically: 


NOT  AT  ALL  NOT  NEUTRAL  SLIGHTLY  EXTREMELY 

USEFUL  USEFUL  USEFUL  USEFUL 

159.  Candidate  Identification  Function,  specifically: 
Between  Failure  Report 

NOT  AT  ALL  NOT  NEUTRAL  SLIGHTLY  EXTREMELY 

USEFUL  USEFUL  USEFUL  USEFUL 


DON’T  KNOW 
Material  Issue 

DON’T  KNOW 
Supply  Synopsis 
DON’T  KNOW 
Wholesale  System 
DON’T  KNOW 
Average  Customer 
DON’T  KNOW 
Backorder  History 
DON’T  KNOW 

NAVICP  NSN  Snapshot 
DON’T  KNOW 
Mean  Flight  Hour 

DON’T  KNOW 
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160.  Candidate  Identification 

Function, 

specifically: 

Wait  Time 

Maintenance 

Impact 

NOT  AT  ALL 
USEFUL 

NOT 

USEFUL 

NEUTRAL 

SLIGHTLY 

USEFUL 

EXTREMELY 

USEFUL 

DON’T  KNOW 

161.  Candidate  Identification 

Function, 

specifically: 

Average  Days  to 

Receipt 

NOT  AT  ALL 
USEFUL 

NOT 

USEFUL 

NEUTRAL 

SLIGHTLY 

USEFUL 

EXTREMELY 

USEFUL 

DON’T  KNOW 

162.  Candidate  Identification 
Actual  Opportunity  Costs 

Function, 

% 

specifically: 

Planned  versus 

NOT  AT  ALL 
USEFUL 

NOT 

USEFUL 

NEUTRAL 

SLIGHTLY 

USEFUL 

EXTREMELY 

USEFUL 

DON’T  KNOW 

163.  Engine 

Repair  Cost 

NOT  AT  ALL 
USEFUL 

NOT 

USEFUL 

NEUTRAL 

SLIGHTLY 

USEFUL 

EXTREMELY 

USEFUL 

DON’T  KNOW 

164.  Flight 

Hours  Since  Last  Engine  Repair 

NOT  AT  ALL 
USEFUL 

NOT 

USEFUL 

NEUTRAL 

SLIGHTLY 

USEFUL 

EXTREMELY 

USEFUL 

DON’T  KNOW 

165.  Engine 

Demand  Forecasting 

NOT  AT  ALL 
USEFUL 

NOT 

USEFUL 

NEUTRAL 

SLIGHTLY 

USEFUL 

EXTREMELY 

USEFUL 

DON’T  KNOW 

166.  Engine 

Overview 

NOT  AT  ALL 
USEFUL 

NOT 

USEFUL 

NEUTRAL 

SLIGHTLY 

USEFUL 

EXTREMELY 

USEFUL 

DON’T  KNOW 

167.  Engine 

Removal 

Trend 

NOT  AT  ALL 
USEFUL 

NOT 

USEFUL 

NEUTRAL 

SLIGHTLY 

USEFUL 

EXTREMELY 

USEFUL 

DON’T  KNOW 

168.  Flight 

Hour  Since  Engine 

Repair  at 

Removal 

NOT  AT  ALL 
USEFUL 

NOT 

USEFUL 

NEUTRAL 

SLIGHTLY 

USEFUL 

EXTREMELY 

USEFUL 

DON’T  KNOW 
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169.  Reference  Information:  Aircraft  Inventory  and  Fatigue  Life 


NOT  AT  ALL  NOT  NEUTRAL  SLIGHTLY  EXTREMELY  DON’T  KNOW 

USEFUL  USEFUL  USEFUL  USEFUL 

170.  Reference  Information:  Code  Definition 

NOT  AT  ALL  NOT  NEUTRAL  SLIGHTLY  EXTREMELY  DON’T  KNOW 

USEFUL  USEFUL  USEFUL  USEFUL 

171.  Reference  Information:  Aircraft  Engine  Installation  and  Ownership 


NOT  AT  ALL  NOT  NEUTRAL  SLIGHTLY  EXTREMELY  DON’T  KNOW 

USEFUL  USEFUL  USEFUL  USEFUL 

172-  Reference  Information:  Production  Load  and  Run  Statistics 

NOT  AT  ALL  NOT  NEUTRAL  SLIGHTLY  EXTREMELY  DON’T  KNOW 

USEFUL  USEFUL  USEFUL  USEFUL 

173.  Reference  Information:  Possible  Courses  of  Action 

NOT  AT  ALL  NOT  NEUTRAL  SLIGHTLY  EXTREMELY  DON’T  KNOW 

USEFUL  USEFUL  USEFUL  USEFUL 

174.  Reference  Information:  Organization  Codes  /Job  Count 

NOT  AT  ALL  NOT  NEUTRAL  SLIGHTLY  EXTREMELY  DON’T  KNOW 

USEFUL  USEFUL  USEFUL  USEFUL 

175.  Reference  Information:  NIIN/CAGE/Part  Number  Cross  Reference 


NOT  AT  ALL  NOT  NEUTRAL  SLIGHTLY  EXTREMELY  DON’T  KNOW 

USEFUL  USEFUL  USEFUL  USEFUL 

176.  Reference  Information:  TEC  Information 

NOT  AT  ALL  NOT  NEUTRAL  SLIGHTLY  EXTREMELY  DON’T  KNOW 

USEFUL  USEFUL  USEFUL  USEFUL 

177.  Reference  Information:  SALTS  File  Information 

NOT  AT  ALL  NOT  NEUTRAL  SLIGHTLY  EXTREMELY  DON’T  KNOW 

USEFUL  '  USEFUL  USEFUL  USEFUL 

178.  Reference  Information:  Data  Dictionary 

NOT  AT  ALL  NOT  NEUTRAL  SLIGHTLY  EXTREMELY  DON’T  KNOW 

USEFUL  USEFUL  USEFUL  USEFUL 
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179.  Other: 


NOT  AT  ALL  NOT  NEUTRAL  SLIGHTLY  EXTREMELY  DON’T  KNOW 

USEFUL  USEFUL  USEFUL  USEFUL 


We  value  your  i.npu't ,  'thank  you  again  for  your  con'tribu'tion  'to  our 
research . 

180.  May  we  contact  you  to  clarify  or  expand  the  information  provided? 

YES  NO 


Please  return  the  questionnaire  by  mail  to  the  following  by  27  March 
1998.  (in  self  addressed  stamped  envelope  provided) 

LCDR  Ellen  Moore 
SMC  1689,  Herman  Hall 
Naval  Postgraduate  School 
Monterey,  CA  93943 

Please  feel  free  to  address  any  questions  or  comments  to: 

LCDR  Ellen  Moore/phone:  408-657-0891/email:  eemoore@nps.navy.mil 
or  LCDR  Carolynn  Snyder/phone:  408-393-9567/email:  cmsnyder@nps.navy.mil 


Please  include  any  additional  comments,  concerns,  or  questions  below  or 
on  attached  pages. 
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APPENDIX  6:  QUESTIONNAIRE  RESULTS 

PART  ONE  (Phone  response  from  LMDSS  Users  and  Non-users) 

A  Identifying  Information: 

1  Name 

2  Phone 

3  email 

4  Address 

5  Job  Trtie 

6  Brief  Description  of  job 

7  Do  you  work  with  new  aviation  systems,  sustaining  existing  systems  or  both? 

new  existing  both 

1  2  5 

B  How  often  do  you  work  with  the  following  tools  to  perform  your  job? 

In  the  capacity  of  identifying  the  requirements,  analyzing  the  requirements,  or  both? 


8  Logistics  Support  Analysis  (MIL-STD-1388) 


never  infrequently 

monthly 

weekly 

daily 

dorit  know 

identify 

analyze 

both 

dont  know 

3  4 

1 

0 

0 

0 

1 

1 

2 

1 

9  Raw  Data 

never  infrequently 

monthly 

weekly 

daily 

dont  know 

identify 

analyze 

both 

dont  know 

0  0 

0 

1 

7 

0 

0 

1 

6 

1 

10  Models 

never  infrequently 

monthly 

weekly 

daily 

don't  know 

identify 

analyze 

both 

dont  know 

4  0 

0 

0 

2 

1 

0 

0 

3 

1 

11  Checklists 

never  infrequently 

monthly 

weekly 

daily 

don’t  know 

identify 

analyze 

both 

dont  know 

3  0 

0 

0 

4 

0 

0 

0 

4 

0 

12  Intuition/Experience 

never  infrequently 

monthly 

weekly 

daily 

dont  know 

identify 

analyze 

both 

dont  know 

0  0 

0 

0 

8 

0 

0 

1 

7 

0 

13  Other 

never  infrequently 

monthly 

weekly 

daily 

dont  know 

identify 

analyze 

both 

dont  know 

0  0 

0 

0 

4 

2 

0 

1 

3 

2 

14  Do  you  know  what  the  LMDSS  is? 

yes  no 

8  0 

15  Have  you  previously  or  do  you  currently  use  the  LMDSS? 

yes  no 

1  7 

16  If  you  use  the  LMDSS,  how  often? 

infrequently  monthly 

weekly 

daily 

dont  know 

0  0 

0 

1 

0 

17  How  often  do  you  interface  with  project  engineer(s)? 

never  infrequently 

monthly 

vreekly 

daily 

dont  know 

identify 

analyze 

both 

dont  know 

1  0 

0 

0 

7 

0 

0 

1 

7- 

0 

18  How  often  do  you  Interfece  with  project  analyst(s)? 

never  infrequently 

monthly 

weekly 

daily 

dont  know 

identify 

analyze 

both 

dont  know 

0  0 

0 

0 

7 

0 

0 

1 

6 

0 

19  How  often  do  you  interface  with  those  who  influence  the  program  budget? 

never  infrequently 

monthly 

weekly 

daily 

dont  know 

identify 

analyze 

both 

dont  know 

0  1 

0 

3 

4 

0 

0 

0 

4 

3 

20  How  often  do  you  interface  with  others? 

never  infrequently 

monthly 

weekly 

daily 

dont  know 

identify 

analyze 

both 

dont  know 

0  0 

0 

2 

6 

0 

0 

0 

6 

0 

21  How  do  you  measure  life  cycle  costs? 
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PART  TWO  (LMDSS  Users) 

C  How  frequently  do  you  work  with  the  following  logistics  concerns? 

Additionally,  indicate  if  it  is  in  the  capacity  of  identifying  requirements,  analyzing  requirements,  or  both. 


22  Logistics  criteria  as  input  to  system  design 
never  infrequently  monthly  weekly 

dally 

don't  know 

identify 

analyze 

both 

don't  know 

0  0  0 

0 

1 

0 

0 

0 

1 

0 

23  Human  factors  concerns 

never  infrequently  monthly 

weekly 

daily 

don't  know 

identify 

analyze 

both 

don't  know 

0  1  0 

0 

0 

0 

0 

0 

1 

0 

24  Failure  mode,  effects, and  ctirical  analysis 
never  infrequently  monthly  weekly 

daily 

don't  know 

identify 

analyze 

both 

don't  know 

0  1  0 

0 

0 

0 

.  0 

0 

1 

0 

25  Failure  reporting,  analysis,  and  corrective  action  system 
never  infrequently  monthly  weekly  daily 

don't  know 

identify 

analyze 

both 

don't  know 

0  10 

0 

0 

0 

0 

0 

1 

0 

26  Provisioning  needs/altematives 

never  infrequently  monthly 

weekly 

daily 

don't  know 

Identify 

analyze 

both 

don't  know 

0  0  0 

0 

1 

0 

0 

0 

1 

0 

27  Compatiblity  with  existing  systems 

never  infrequently  monthly 

weekly 

daily 

don't  know 

identify 

analyze 

both 

dont  know 

0  0  0 

0 

1 

0 

0 

0 

1 

0 

28  Configuration  Management 

never  infrequently  monthly 

weekly 

daily 

don't  know 

identify 

analyze 

both 

don't  know 

0  10 

0 

0 

0 

0 

0 

1 

0 

29  Training  and  training  support 

never  infrequently  monthly 

weekly 

daily 

don't  know 

identify 

analyze 

both 

dont  know 

0  0  0 

0 

1 

0 

0 

0 

1 

0 

30  Manpower  and  personnel 

never  infrequentiy  monthly 

weekly 

daily 

don't  know 

identify 

analyze 

both 

dont  know 

0  0  0 

0 

1 

0 

0 

0 

1 

0 

31  Supply  support/spares 

never  infrequently  monthly 

weekly 

daily 

don't  know 

Identify 

analyze 

both 

dont  know 

0  0  0 

0 

1 

0 

0 

0 

1 

0 

32  Inventory  level  analysis 

never  infrequently  monthly 

weekly 

daily 

don't  know 

identify 

analyze 

both 

dont  know 

0  0  0 

0 

1 

0 

0 

0 

1 

0 

33  Transportation,  packaging  or  storage 

never  infrequentiy  monthly 

weekly 

daily 

don't  know 

identify 

analyze 

both 

dont  know 

0  1  0 

0 

0 

0 

.  0 

0 

1 

0 

34  Test  equipment 

never  infrequently  monthly 

weekly 

daily 

don't  know 

identify 

analyze 

both 

dont  know 

0  0  0 

0 

1 

0 

0 

0 

1 

0 

35  Support  equipment 

never  infrequently  monthly 

weekly 

daily 

don't  know 

identify 

analyze 

both 

dont  know 

0  0  0 

0 

1 

0 

0 

0 

1 

0 

36  Computer  Resource  support 

never  infrequently  monthly 

weekly 

daily 

don't  know 

identify 

analyze 

both 

dont  know 

0  1  0 

0 

0 

0 

0 

0 

1 

0 

37  Facilities,  requirements 

never  infrequently  monthly 

weekly 

daily 

don't  know 

Identify 

analyze 

both 

dont  know 

0  1  0 

0 

0 

0 

0 

0 

1 

0 

38  Facilities,  location 

never  Infrequently  monthly 

weekly 

daily 

don't  know 

identify 

analyze 

both 

dont  know 

0  1  0 

0 

0 

0 

0 

0 

1 

0 

39  Data,  reports  requirements 

never  infrequently  monthly 

Vi^kly 

daily 

don't  know 

identify 

analyze 

both 

dont  know 

0  0  0 

0 

1 

0 

0 

0 

1 

0 
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40  Maintenance  planning  (scheduled  versus  unscheduled  plan) 


never  infrequently  monthly 

weekly 

daily 

dont  know 

identify 

analyze 

both 

0  0  0 

0 

1 

0 

0 

0 

1 

41  Level  of  repair  analysis  (0  versus  1  versus  D-Ievels) 

never  infrequently  monthly 

weekly 

daily 

dont  know 

identity 

analyze 

both 

0  10 

0 

0 

0 

0 

0 

1 

42  Operating  environment  issues 

never  infrequently  monthly 

weekly 

daily 

dont  know 

identity 

analyze 

both 

0  1  0 

0 

0 

0 

0 

0 

1 

43  Cost-drivers 

never  infrequently  monthly 

weekly 

daily 

dont  know 

identity 

analyze 

both 

0  0  0 

0 

1 

0 

.  0 

0 

1 

44  Readiness  degraders 

never  infrequently  monthly 

weekly 

daily 

dont  know 

identity 

analyze 

both 

0  0  0 

0 

1 

0 

0 

0 

1 

45  Cycle  time  to  repair  components 

never  infrequently  monthly 

weekly 

daily 

dont  know 

identity 

analyze 

both 

0  0  0 

0 

1 

0 

0 

0 

1 

46  Other 

never  infrequently  monthly 

weekly 

daily 

dont  know  ' 

identity 

analyze 

both 

0  0  0 

0 

0 

0 

0 

0 

0 

47  How  often  do  you  use  LMDSS? 

never  infrequently  monthly 

weekly 

daily 

dont  know 

0  0  0 

0 

1 

0 

48  Summary  data  for  end  items 

never  infrequently  monthly 

weekly 

daily 

dont  know 

0  0  0 

0 

1 

0 

49  Reliability  summary  parameters 

never  infrequently  monthly 

weekly. 

daily 

dont  know 

0  0  0 

0 

1 

0 

50  Supportability  summary  parameters 

never  infrequently  monthly 

weekly 

daily 

dont  know 

0  0  0 

0 

1 

0 

51  Cost  summary  parameters 

never  Infrequently  monthly 

weekly 

daily 

dont  know 

0  0  0 

0 

1 

0 

52  Trend  analysis  (problems  and  causes) 

never  infrequently  monthly 

weekly 

dally 

dont  know 

0  0  0  0  1  0 

53  Component  and/or  end-ltem  cost  data,  specifically:  Annual  Operating  and  Support  Costs 

never  infrequently  monthly  weekly  ^  daily  don^t  know 

0  0  0.0  1  0 

54  Component  and/or  end-item  cost  data,  specifically:  Labor  Cost  History 

never  infrequently  monthly  weekly  daily  don*t  know 

0  0  0  0  1  0 

55  Component  and/or  end-item  cost  data,  specifically:  Item  Value  to  Depot  Repair  Cost 

never  infrequently  monthly  weekly  daily  dont  know 

0  0  0  0  1  0 

56  Component  and/or  end-item  cost.data,  specifically:  Budget  Analysis  (OP-20  Report) 

never  infrequently  monthly  weekly  daily  dont  know 

0  1  0  0  0  0 

57  Component  and/or  end-item  cost  data,  specifically:  Inflation  Factors 

never  infrequently  monthly  weekly  daily  dont  know 

0  1  0  0  0  0 

58  Component  and/or  end-item  cost  data,  specifically:  Item  Value  to  Labor  Cost 

never  infrequently  monthly  weekly  daily  dont  know 

0  1  0  0  0  0 


dont  know 
0 

dont  know 
0 

dont  know 
0 

dont  know 
0 

dont  know 
0 

dont  know 
0 

dont  know 
0 
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59  Candidate  Identification  Function,  specrficany:  Detail  Component  Report 

never  infrequently  monthly  weekly  daily  dont  know 

0  0  0  0  1  0 

60  Candidate  Identification  Function,  specifically:  Wholesale  System  Demand 

never  *  infrequently  monthly  v/eekly  daily  don't  know 

0  1  0  0  0  0 

61  Candidate  Identification  Function,  specifically:  Material  Issue  Trends 

never  infrequently  monthly  weekly  daily  dontknow 

0  0  0  0  1  0 

62  Candidate  Identification  Function,  specrficany:  Supply  Synopsis 

never  infrequently  monthly  weekly  daily  dont  know 

0  1  0  0  0  0 

63  Candidate  Identification  Function,  specifically:  Wholesale  System  Investment 

never  Infrequently  monthly  weekly  daily  dont  know 

0  1  0  0  0  0 

64  Candidate  Identification  Function,  specifically:  Average  Customer  Wart  Reports 

never  infrequently  monthly  weekly  daily  dont  know 

0  1  0  0  0  0 

65  Candidate  Identification  Function,  specificaliy:  Backorder  History  Reports 

never  infrequently  monthly  weekly  daily  dont  know 

0  1  0  0  0  0 

66  Candidate  Identification  Function,  specifically:  NUN  NSN  Snapshots 

never  infrequently  monthly  >weekly  daily  dont  know 

0  0  0  0  1  0 

67  Candidate  Identification  Function,  specificaIIy:Mean  Flight  Hour  Between  Failure  Report 


never  infrequently 

monthly 

>veekly 

daily 

dont  know 

0  0 

0 

0 

1 

0 

68  Engine  Repair  Cost 

never  infrequently 

monthly 

weekly 

daily 

dont  know 

0  1 

0 

0 

0 

0 

69  Flight  Hours  Since  Last  Engine  Repair 

never  infrequently 

monthly 

weekly 

daily 

dpn*t  krww 

0  1 

0 

0 

0 

0 

70  Engine  Demand  Forecasting 

never  infrequently 

monttily 

weekly 

daily 

dont  know 

0  1 

0 

0 

0 

0 

71  Engine  Overview 

never  infrequently 

monthly 

weekly 

daily 

dont  know 

0  1 

0 

0 

0 

0 

72  Reference  Information:  Aircraft  Inventory  and  Fatigue  Life 

never  infrequently 

monthly 

weekly 

daily 

dont  know 

0  1 

0 

0 

0 

0 

73  Reference  Information:  Code  Definition 

never  infrequently 

monthly 

vi^kly 

daily 

dont  know 

0  1 

0 

0 

0 

0 

74  Reference  Information:  Aircraft  Engine  Installation  and  Ownership 

never  infrequerrtly 

monthly 

weekly 

daily 

don't  know 

0  1 

0 

0 

0 

0 

75  Reference  Information:  Production  Load  and  Run  Statistics 

never  infrequently 

monthly 

weekly 

daily 

dont  know 

0  1 

0 

0 

0 

0 

76  Reference  Information:  Possible  Courses  of  Action 

never  infrequently 

monthly 

weekly 

daily 

dont  know 

0  1 

0 

0 

0 

0 

77  Reference  Information:  Organization  Codes/Job  Count 

never  infrequently 

monthly 

weekly 

daily 

dont  know 

0  0 

0 

0 

1 

0 
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78  Reference  Informaton:  NIIN/CAGE/Part  Number  Cross  Reference 


never  infrequently  monthly 

weekly 

daily 

don't  know 

0  0  0  .0 

79  Reference  Information:  TEC  Information 

1 

0 

never  infrequently  monthly 

weekly 

daily 

don't  know 

0  0  0  0 

80  Reference  Information:  SALTS  File  Information 

1 

0 

never  infrequently  monthly 

weekly 

daily 

don't  know 

0  10  0 

81  Reference  Information:  Data  Dictionary 

b 

0 

never  infrequently  monthly 

weekly 

daily 

don't  know 

0  1  0 

0 

0 

0 

E  Please  describe  your  experience  with  the  following  LMDSS  functions. 

82  Interface 

strongly  dislike  neutral 

dislike 

nke 

strongly 

like 

don't  know 

0  0  0 

1 

0 

0 

83  Help  Function 

strongly  dislike  neutral 

dislike 

like 

strongly 

like 

don't  know 

0  0  0 

1 

0 

0 

84  Analysis  Tools 

strongly  dislike  neutral 

dislike 

like 

strongly 

like 

don't  know 

0  0  0 

1 

0 

0 

85  Report  presentation/fonmat 

strongly  dislike  neutral 

dislike 

like 

strongly 

like 

don't  know 

0  0  0 

1 

0 

0 

86  Time  required  getting  what  is  needed 

strongly  dislike  neutral 

dislike 

like 

strongly 

like 

don't  know 

0  1  0 

87  Ease  of  getting  what  is  needed 

0 

0 

0 

strongly  dislike  neutral 

dislike 

like 

strongly 

like 

dont  know 

0  10 

0 

0 

0 

88  Training 

strongly  dislike  neutral 

dislike 

like 

strongly 

like 

dont  know 

1  0  0 

0 

0 

0 

89  Accessibility  (when  desired) 

strongly  dislike  neutral 

dislike 

like 

strongly 

like 

dont  know 

0  0  1 

0 

0 

0 

90  Accessibility  (server  access) 

strongly  dislike  neutral 

dislike 

like 

strongly 

like 

dont  know 

0  0  1 

91  Accessibifity  (password  access) 

0 

0 

0 

strongly  dislike  neutral 

dislike 

like 

strongly 

like 

dont  know 

0  0  0 

1 

0 

0 

92  Provides  the  information  1  need 

strongly  dislike  neutral 

dislike 

like 

strongly 

like 

dont  know 

0  0  0 

1 

0 

0 
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F  Please  describe  your  experience  using  the  LMDSS  data  currently  available. 


93  Data  meets  my  needs 

strongly  disagree 

disagree 

neutral 

agree 

strongly 

agree 

don't  know 

0  0 

0 

1 

0 

0 

94  Data  is  accessible 

strongly  disagree 

disagree 

neu^I 

agree 

strongly 

agree 

dont  know 

0  0  0 

95  Data  is  accurate/consistent 

1 

0 

0 

strongly  disagree 

disagree 

neutral 

agree 

strongly 

agree 

don't  know 

1  0 

0 

0 

0 

0 

96  Data  is  detailed  enough 

strongly  disagree 

disagree 

neutral 

agree 

strongly 

agree 

dont  know 

0  0  0 

97  The  exact  data  meaning  is  clear 

1 

0 

0 

strongly  disagree 

disagree 

neutral 

agree 

strongly 

agree 

don't  know 

0  1 

0 

0 

0 

0 

G  Please  descn*be  your  experience  using  other  data  currently  available. 
98  Data  meets  my  needs 


strongly  disagree 

disagree 

neutral 

agree 

strongly 

agree 

dont  know 

0  0 

0 

0 

1 

0 

99  Data  is  accessible 

strongly  disagree 

disagree 

neutral 

agree 

strongly 

agree 

dont  know 

0  0  0 

100  Data  is  accurate/consistent 

0 

1 

0 

strongly  disagree 

disagree 

neutral 

agree 

strongly 

agree 

dont  know 

0  0 

0 

0 

1 

0 

101  Data  is  detailed  enough 

strongly  disagree 

disagree 

neutral 

agree 

strongly 

agree 

dont  know 

0  0  0 

102  The  exact  data  meaning  is  dear 

0 

1 

0 

strongly  disagree 

disagree 

neutral 

agree 

strongly 

agree 

dont  know 

0  0  0 
PART  THREE  (LMDSS  Nonnisers) 

0 

1 

0 

H  How  frequently  do  you  work  with  the  following  logistics  concerns? 


Additionally,  indicate  if  it  is  in  the  capacity  of  identifying  requirements,  analyzing  requirements,  both. 
103  Logistics  criteria  as  input  to  system  design 


never  Infrequently  monthly 

weekly 

daily 

dont  know 

identify 

analyze 

both 

dont  know 

0  3  2 

0 

2 

0 

2 

1 

4 

0 

104  Human  factors  concerns 

never  infrequently  monthly 

vreekly 

daily 

dont  know 

identify 

analyze 

both 

dont  know 

0  4  1 

0 

2 

0 

1 

3 

3 

0 

105  Failure  mode,  effects,and  ctirical  analysis 

never  .  infrequently  monthly 

weekly 

daily 

dont  know 

identify 

analyze 

both 

dont  know 

0  4  0 

1 

2 

0 

1 

2 

4 

0 

106  Failure  reporting,  analysis,  and  comective  action  system 

never  infrequently  monthly 

weekly 

daily 

dont  know 

identify 

analyze 

both 

dont  know 

0  1  2 

2 

1 

1 

1 

1 

4 

1 
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107  Provisioning  needs/altematives 


never  infrequently  monthly 

v/eekly 

daily 

don't  know 

identify 

arralyze 

both 

don't  know 

0  1  2 

2 

1 

1 

1 

1 

4 

1 

108  Compatiblity  with  existing  systems 

never  infrequently  monthly 

weekly 

daily 

don’t  know 

identify 

analyze 

both 

don’t  know 

0  2  1 

1 

2 

0 

0 

2 

3 

1 

109  Configuration  Management 

never  infrequently  montiily 

weekly 

daily 

don’t  know 

identify 

analyze 

both 

don’t  know 

0  2  0 

2 

3 

0 

0 

2 

6 

0 

110  Training  and  training  support 

never  infrequently  monthly 

weekly 

daily 

don’t  know 

identify 

analyze 

both 

don't  know 

0  1  3 

2 

1 

0 

.  1 

1 

5 

0 

111  Manpower  and  personnel 

never  infrequently  monthly 

weekly 

daily 

don’t  know 

identify 

analyze 

botii 

don't  know 

0  3  4 

0 

0 

0 

2 

3 

2 

0 

112  Supply  support/spares 

never  Infrequently  monthly 

weekly 

daily 

don’t  know 

Identify 

analyze 

both 

don't  know 

0  1  0 

3 

3 

0 

0 

1 

6 

0 

113  Inventory  level  analysis 

never  infrequently  monthly 

vi/eekly 

daily 

don’t  know 

identify 

analyze 

both 

don’t  know 

0  2  2 

2 

1 

0 

0 

1 

5 

1 

114  Transportation,  packaging  or  storage 

never  infrequently  monthly 

weekly 

daily 

don't  know 

identify 

analyze 

both 

don't  know 

0  4  0 

2 

1 

0 

1 

1 

4 

1 

116  Test  equipment 

never  infrequently  monthly 

weekly 

dally 

don’t  know 

identify 

analyze 

both 

don’t  know 

0  1  3 

2 

1 

*  0 

1 

1 

5 

0 

1 1 6  Support  equipment 

never  infrequently  monthly 

weekly 

daily 

don’t  know 

identify 

analyze 

both 

don't  know 

0  1  2 

2 

2 

0 

1 

1 

5 

0 

117  Computer  Resource  support 

never  infrequently  monthly 

weekly 

daily 

don’t  know 

identify 

analyze 

both 

don’t  know 

.0  2  2 

2 

1 

0 

2 

1 

3 

1 

118  Facilities,  requirements 

never  infrequently  monthly 

weekly 

daily 

don’t  know 

identify 

analyze 

both 

don’t  know 

0  4  2 

1 

0 

0 

0 

2 

5 

0 

119  Facilities,  location 

never  infrequently  monthly 

weekly 

daily 

don’t  know 

identify 

analyze 

both 

don’t  know 

0  5  2 

0 

0 

0 

2 

2 

2 

1 

120  Data,  reports  requirements 

never  infrequently  monthly 

weekly 

daily 

don't  know 

identify 

analyze 

both 

don’t  know 

0  0  1 

2 

4 

0 

0 

1 

6 

0 

121  Maintenance  planning  (scheduled  versus  unscheduled  plan) 

never  infrequently  monthly  weekly  daily  don't  know 

identify 

analyze 

both 

don't  know 

1  2  0 

2 

2 

0 

0 

2 

4 

0 

1 22  Level  of  repair  analysis  (0  versus  1  versus  D-levels) 

never  infrequently  monthly  weekly  daily 

don't  know 

identify 

analyze 

both 

don’t  know 

0  3  0 

2 

2 

0 

0 

1 

6 

0 

1 23  Operating  environment  issues 

never  infrequently  monthly 

weekly 

daily 

don’t  know 

identify 

analyze 

both 

don’t  know 

0  2  2 

2 

1 

0 

0 

2 

5 

0 

124  Cost-drivers 

never  infrequently  monthly 

weekly 

daily 

don’t  know 

identify 

analyze 

both 

don’t  know 

0  2  3 

0 

2 

0 

0 

2 

5 

0 

125  Readiness  degraders 

never  infrequently  montiily 

weekly 

daily 

don’t  know 

identify 

analyze 

both 

don’t  know 

0  2  1 

1 

3 

0 

1 

1 

5 

0 
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126  Cycle  time  to  repair  components 
never  infrequently  monthly 

weekly 

daily 

dont  know 

identify 

analyze 

both 

dont  know 

0  3  2 

1 

1 

0 

0 

1 

5 

1 

127  Other 

never  infrequently  monthly 

v^kly 

daily 

dont  know 

identify 

analyze 

botii 

dont  know 

0  0  1 

0 

1 

0 

0 

0 

2 

0 

1 28  How  often  do  you  use  LMDSS? 
never  infrequently  monthly 

7  0  0 

weekly 

0 

daily 

0 

dont  know 

0 

129  Why  do  you  not  use  LMDSS?  (check  all  that  apply) 

2  I  didnt  know  it  existed 

1  My  PC  won't  support  LMDSS 

4  I've  received  no  training 

0  I  dont  need  the  information  LMDSS  provides 
0  It  takes  too  tong  to  get  what  I  need 
0  Itstoodlfficulttogetwhat  I  need 

2  It  doesn't  provide  me  the  information  I  need 

1  other 


I  Please  describe  your  experience  using  other  data  currentiy  available. 


130  Data  meets  my  needs 

strongly  disagree 

disagree 

neutral 

agree 

strongly 

agree 

dont  know 

0  0 

3 

3 

0 

0 

131  Data  is  accessible 

strongly  disagree 

disagree 

neutral 

agree 

strongly 

agree 

dont  know 

0  0  2 

132  Data  is  accurate/consistent 

3 

1 

0 

strongly  disagree 

disagree 

neutral 

agree 

strongly 

agree 

dont  know 

0  2 

2 

1 

0 

1 

1 33  Data  is  detailed  enough 

strongly  disagree 

disagree 

neutral 

agree 

strongly 

agree 

dont  know 

0  2  2 

134  The  exact  data  meaning  is  clear 

2 

0 

0 

strongly  disagree 

disagree  . 

neutral 

agree 

strongly 

agree 

dont  know 

0  2 

3 

1 

0 

0 

PART  FOUR  (Job  Information  Needs,  LMDSS  Users  and  Non-users) 

J  Indicate  how  useful  the  following  information  elements  are  (or  Nvould  be)  in  performing  your  job. 

135  Summary  Data;  End  Item 

not  at  all  not  neutral  slightly  extremely  dont  know 

useful  useful  useful  useful 

0  0  0  2  4  1 

136  Summary  Data;  Claimant 

not  at  all  not  neutral  slightly  extremely  dont  know 

useful  useful  useful  useful 

0  0  2  1  2  2 

137  Summary  Data;  Organization 

not  at  all  not  neutral  slightly  exfremely  dont  know 

useful  useful  useful  useful 

0  10  1  5  0 

138  Summary  Data;  BCM  Report 

not  at  all  not  neutal  slightly  extremely  dont  know 

useful  useful  useful  useful 

0  1  0  2  4  0 
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139  Reliability  summary  parameters 


not  at  all  not 

neutral 

slightly 

extremely  don't  know 

useful  useful 

useful 

useful 

0  0 

0 

1 

6  0 

140  Supportability  summary  parameters 

not  at  all  not 

neutral 

slightly 

extremely  don't  know 

useful  useful 

useful 

useful 

b  0 

0 

1 

5  1 

141  Cost  summary  parameters 

not  at  all  not 

neutral 

slightly 

extremely  don't  know 

useful  useful 

useful 

useful 

0  0 

0 

1 

6  0 

142  Emerging  Problems 

not  at  all  not 

neutral 

slightly 

extremely  don't  know 

useful  useful 

useful 

useful 

0  0 

0 

0 

7  0 

143  Common  Equipment 

not  at  all  not 

neutral 

slightly 

extremely  don't  know 

useful  useful 

useful 

useful 

0  0 

0 

1 

4  2 

144  Trend  Analysis  (problems  versus  causes) 

not  at  all  not 

neutral 

slightly 

extremely  don't  know 

useful  useful 

useful 

useful 

0  0 

0 

1 

5  1 

145  Component  and/or  end-item  cost  data,  specifically:  Annual  Operating  and  Support  Costs 

not  at  all  not 

neutral 

slightly 

extremely  don't  know 

useful  useful 

useful 

useful 

0  0 

0 

0 

7  0 

146  Component  and/or  end-item  cost  data,  specifically:  Labor  Cost  History 

not  at  all  not 

neutral 

slightly 

extremely  don't  know 

useful  useful 

useful 

useful 

0  0 

0 

0 

7  0 

147  Component  and/or  end-item  cost  data,  specifically:  Item  Value  to  Depot  Repair  Cost 

not  at  all  not 

neutral 

slightly 

extremely  don't  know 

useful  useful 

useful 

useful 

0  0 

0 

0 

5  2 

148  Component  and/or  end-item  cost  data,  specifically:  Budget  Analysis  (OP-20  Report) 

not  at  all  not 

neutral 

slightly 

extremely  don't  know  ■ 

useful  useful 

useful 

useful 

0  0 

1 

2 

3  1 

149  Component  and/or  end-item  cost  data,  specifically:  Inflation  Factors 

not  at  all  not 

neutral 

slightly 

extremely  dontknow 

useful  useful 

useful 

useful 

0  0 

1 

3 

2  1 

150  Component  and/or  end-item  cost  data,  specifically:  Item  Value  to  Labor  Cost 

not  at  ail  not 

neutral 

slightly 

extremely  dontknow 

useful  useful 

useful 

useful 

0  0 

0 

2 

3  2 

151  Candidate  Identification  Function,  specifically:  Detail  Component  Report 

not  at  all  not 

neutral 

slightiy 

extremely  dontknow 

useful  useful 

useful 

useful 

0  0 

2 

0 

4  1 

152  Candidate  Identification  Function,  specifically:  Wholesale  System  Demand 

not  at  all  not 

neutral 

slightiy 

extremely  dontknow 

useful  useful 

useful 

useful 

0  0 

1 

4 

11 

149 


153  Candidate  Identification  Function,  specifically:  Materia!  Issue  Trends 

not  at  all  not  neutral  slightly  extremely  don't  know 

useful  useful  ^  useful  useful 

0  0  1  2  3  1 

154  Candidate  Identification  Function,  specifically:  Supply  Synopsis 

not  at  all  not  neutral  slightiy  extremely  don't  know 

useful  useful  useful  useful 

0  0  1  1  4  1 

155  Candidate  Identification  Function,  specifically:  Wholesale  System  Investment 

not  at  all  not  neutral  slightly  extremely  don't  know 

useful  useful  useful  useful 

0  0  1  3  2  1 

156  Candidate  Identification  Function,  specifically:  Average  Customer  Wart  Reports 

not  at  all  not  neutral  slightiy  extremely  don't  know 

useful  useful  useful  useful 

0  0  1  3  3  0 

157  Candidate  Identification  Function,  specifically:  Backorder  History  Reports 

not  at  all  not  neutral  slightly  extremely  don't  know 

useful  useful  useful  useful 

0  1  1  2  3  0 

158  Candidate  Identification  Function,  specifically:  NIIN  NSN  Snapshots 

not  at  all  not  neutral  slightiy  extremely  don't  know 

useful  useful  useful  useful 

0  1  0  2  4  0 

159  Candidate  Identification  Function,  specifically: Mean  Flight  Hour  Between  Failure  Report 

not  at  all  not  neutral  slightly  extremely  don't  know 

useful  useful  useful  useful 

0  0  0  1  5  0 

160  Candidate  Identification  Function,  specifically: Wait  Time  Maintenance  Impact 

not  at  all  not  neutral  slightly  extremely  don't  know 

useful  useful  useful  useful 

0  0  1  2  3  1 

161  Candidate  Identification  Function,  specifically: Average  Days  to  Receipt 

not  at  all  not  neutral  slightly  extremely  don't  know 

useful  useful  useful  useful 

0  0  1  3  3  0 

162  Candidate  Identification  Function,  specificany: Planned  versus  Actual  Opperating  Costs 


not  at  all 

not 

neutral 

slightly 

extremely 

don't  know 

useful 

useful 

u^ful 

useful 

0 

0 

1 

2 

2 

2 

163  Engine  Repair  Cost 

not  at  all 

not 

neutral 

slightly 

extremely 

don't  know 

useful 

useful 

useful 

useful 

0 

0 

0 

0 

5 

1 

164  Flight  Hours  Since  Last  Engine  Repair 

not  at  all 

not 

neutral 

slightly 

extremely 

don't  know 

useful 

useful 

useful 

useful 

0 

0 

0 

1 

4 

1 

165  Engine  Demand  Forecasting 

not  at  all 

not 

neutral 

slightly 

extremely 

don't  know 

useful 

useful 

useful 

useful 

0 

0 

1 

0 

4 

1 

166  Engine  Overview 

not  at  all 

not 

neutral 

slightly 

extremely 

don't  know 

useful 

useful 

useful 

useful 

0 

0 

1 

1 

2 

2 

150 


167  Engine  Removal  Trend 

not  at  all 

not 

neutral 

slightly 

extremely 

don't  know 

useful 

useful 

useful 

useful 

0 

0 

1 

0. 

4 

1 

168  Flight  Hours  Since  Last  Engine  Repair  at  Removal 

not  at  all 

not 

neutral 

slightly 

extremely 

dont  know 

useful 

useful 

useful 

useful 

0 

0 

1 

0 

3 

2 

169  Reference  Information:  Aircraft  Inventory  and  Fatigue  Life 

not  at  all 

not 

neutral 

slightly 

extremely 

dont  know 

useful 

useful 

useful 

useful 

0 

1 

1 

1 

3 

0 

170  Reference  Information:  Code  Definition 

not  at  all 

not 

neutral 

slightly 

extremely 

don't  know 

useful 

useful 

useful 

useful 

0 

0 

1 

2 

2 

1 

171  Reference  Information:  Aircraft  Engine  Installation  and  Ownership 

not  at  all 

not 

neutral 

slightly 

extremely 

don't  know 

useful 

useful 

useful 

useful 

0 

1 

1 

1 

2 

1 

172  Reference  Information:  Production  Load  and  Run  Statistics 

not  at  all 

not 

neutral 

slightly 

extremely 

don't  know 

useful 

useful 

useful 

useful 

0 

0 

1 

4 

1 

1 

173  Reference  Information:  Possible  Courses  of  Action 

not  at  ail 

not 

neutral 

slightly 

extremely 

dont  know 

useful 

useful 

useful 

useful 

0 

0 

1 

0 

4 

2 

174  Reference  Information:  Organization  Codes/Job  Count 

not  at  all 

not 

neutral 

slightiy 

extremely 

don't  know 

useful 

useful 

useful 

useful 

0 

1 

2 

2 

2 

0 

175  Reference  Information:  NIIN/CAGE/Part  Number  Cross  Reference 

not  at  all 

not 

neutral 

slightly 

extremely 

don't  know 

useful 

useful 

useful 

useful 

0 

0 

1 

2 

4 

0 

176  Reference  Information:  TEC  Information 

not  at  all 

not 

neutral 

slightly 

extremely 

don't  know 

useful 

useful 

useful 

useful 

0 

0 

1 

3 

3 

0 

177  Reference  Information:  SALTS  File  Information 

not  at  all 

not 

neutral 

slightiy 

extremely 

don't  know 

useful 

useful 

useful 

useful 

1 

0 

3 

1 

0 

2 

178  Reference  Information:  Data  Dictionary 

rK>t  at  all 

not 

neutral 

slightiy 

extremely 

don't  know 

useful 

useful 

useful 

useful 

0 

0 

0 

4 

1 

~  2 

179  Other 

not  at  all 

not 

neutral 

slightly 

extremely 

don't  know 

useful  . 

useful 

useful 

useful 

0 

0 

0 

0 

0 

0 

1 80  May  we  contact  you  to  clarify  or  expand  on  the  information  provided? 
yes  no 

8  0 
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